Sep 18 06:20:13.159689 ip-10-0-128-30 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 18 06:20:13.159703 ip-10-0-128-30 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 18 06:20:13.159712 ip-10-0-128-30 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 18 06:20:13.159908 ip-10-0-128-30 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 18 06:20:23.395520 ip-10-0-128-30 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 18 06:20:23.395537 ip-10-0-128-30 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot aea172269419497eaed2880fa4353c37 --
Sep 18 06:22:12.654594 ip-10-0-128-30 systemd[1]: Starting Kubernetes Kubelet...
Sep 18 06:22:16.313912 ip-10-0-128-30 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 06:22:16.313912 ip-10-0-128-30 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 18 06:22:16.313912 ip-10-0-128-30 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 06:22:16.313912 ip-10-0-128-30 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 18 06:22:16.313912 ip-10-0-128-30 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 06:22:16.314798 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.313982    2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 18 06:22:16.317599 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317582    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 06:22:16.317599 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317598    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 06:22:16.317667 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317603    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 06:22:16.317667 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317607    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 06:22:16.317667 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317610    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 06:22:16.317667 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317614    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 06:22:16.317667 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317617    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 06:22:16.317667 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317620    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 06:22:16.317667 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317623    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 06:22:16.317667 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317626    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 06:22:16.317667 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317629    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 06:22:16.317667 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317634    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 06:22:16.317667 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317638    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 06:22:16.317667 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317641    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 06:22:16.317667 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317644    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 06:22:16.317667 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317647    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 06:22:16.317667 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317649    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 06:22:16.317667 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317652    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 06:22:16.317667 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317655    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 06:22:16.317667 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317658    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 06:22:16.317667 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317666    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 06:22:16.317667 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317669    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 06:22:16.318150 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317672    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 06:22:16.318150 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317675    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 06:22:16.318150 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317678    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 06:22:16.318150 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317680    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 06:22:16.318150 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317683    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 06:22:16.318150 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317686    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 06:22:16.318150 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317688    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 06:22:16.318150 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317691    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 06:22:16.318150 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317693    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 06:22:16.318150 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317696    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 06:22:16.318150 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317699    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 06:22:16.318150 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317701    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 06:22:16.318150 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317704    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 06:22:16.318150 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317706    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 06:22:16.318150 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317709    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 06:22:16.318150 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317711    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 06:22:16.318150 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317714    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 06:22:16.318150 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317717    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 06:22:16.318150 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317719    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 06:22:16.318150 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317721    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 06:22:16.318658 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317725    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 06:22:16.318658 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317729    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 06:22:16.318658 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317731    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 06:22:16.318658 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317734    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 06:22:16.318658 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317736    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 06:22:16.318658 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317739    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 06:22:16.318658 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317741    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 06:22:16.318658 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317744    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 06:22:16.318658 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317747    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 06:22:16.318658 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317749    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 06:22:16.318658 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317752    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 06:22:16.318658 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317755    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 06:22:16.318658 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317757    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 06:22:16.318658 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317761    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 06:22:16.318658 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317765    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 06:22:16.318658 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317768    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 06:22:16.318658 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317770    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 06:22:16.318658 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317773    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 06:22:16.318658 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317776    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 06:22:16.318658 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317779    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 06:22:16.319156 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317781    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 06:22:16.319156 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317785    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 06:22:16.319156 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317789    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 06:22:16.319156 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317792    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 06:22:16.319156 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317795    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 06:22:16.319156 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317797    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 06:22:16.319156 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317800    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 06:22:16.319156 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317802    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 06:22:16.319156 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317805    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 06:22:16.319156 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317808    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 06:22:16.319156 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317810    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 06:22:16.319156 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317813    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 06:22:16.319156 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317815    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 06:22:16.319156 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317818    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 06:22:16.319156 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317821    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 06:22:16.319156 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317823    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 06:22:16.319156 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317827    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 06:22:16.319156 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317830    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 06:22:16.319156 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317832    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 06:22:16.319666 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317834    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 06:22:16.319666 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317837    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 06:22:16.319666 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317840    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 06:22:16.319666 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317843    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 06:22:16.319666 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317845    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 06:22:16.319666 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317848    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 06:22:16.319666 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317851    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 06:22:16.319666 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317854    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 06:22:16.319666 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317857    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 06:22:16.319666 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317860    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 06:22:16.319666 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317863    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 06:22:16.319666 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317865    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 06:22:16.319666 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317868    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 06:22:16.319666 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317871    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 06:22:16.319666 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317873    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 06:22:16.319666 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317875    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 06:22:16.319666 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317878    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 06:22:16.319666 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.317880    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 06:22:16.319666 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318594    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 06:22:16.320130 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318601    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 06:22:16.320130 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318604    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 06:22:16.320130 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318607    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 06:22:16.320130 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318609    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 06:22:16.320130 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318612    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 06:22:16.320130 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318615    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 06:22:16.320130 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318618    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 06:22:16.320130 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318620    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 06:22:16.320130 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318623    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 06:22:16.320130 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318625    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 06:22:16.320130 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318629    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 06:22:16.320130 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318632    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 06:22:16.320130 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318634    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 06:22:16.320130 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318637    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 06:22:16.320130 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318639    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 06:22:16.320130 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318642    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 06:22:16.320130 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318645    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 06:22:16.320130 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318647    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 06:22:16.320130 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318650    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 06:22:16.320130 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318654    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 06:22:16.320643 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318657    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 06:22:16.320643 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318660    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 06:22:16.320643 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318662    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 06:22:16.320643 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318665    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 06:22:16.320643 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318668    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 06:22:16.320643 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318670    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 06:22:16.320643 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318673    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 06:22:16.320643 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318676    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 06:22:16.320643 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318678    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 06:22:16.320643 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318681    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 06:22:16.320643 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318683    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 06:22:16.320643 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318686    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 06:22:16.320643 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318689    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 06:22:16.320643 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318691    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 06:22:16.320643 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318694    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 06:22:16.320643 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318696    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 06:22:16.320643 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318699    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 06:22:16.320643 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318702    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 06:22:16.320643 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318704    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 06:22:16.320643 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318707    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 06:22:16.321135 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318709    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 06:22:16.321135 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318712    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 06:22:16.321135 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318715    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 06:22:16.321135 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318717    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 06:22:16.321135 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318720    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 06:22:16.321135 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318722    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 06:22:16.321135 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318724    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 06:22:16.321135 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318727    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 06:22:16.321135 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318729    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 06:22:16.321135 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318734    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 06:22:16.321135 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318737    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 06:22:16.321135 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318743    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 06:22:16.321135 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318747    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 06:22:16.321135 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318750    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 06:22:16.321135 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318752    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 06:22:16.321135 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318755    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 06:22:16.321135 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318757    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 06:22:16.321135 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318760    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 06:22:16.321135 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318763    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 06:22:16.321135 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318766    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 06:22:16.321635 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318768    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 06:22:16.321635 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318770    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 06:22:16.321635 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318773    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 06:22:16.321635 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318776    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 06:22:16.321635 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318778    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 06:22:16.321635 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318781    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 06:22:16.321635 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318784    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 06:22:16.321635 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318786    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 06:22:16.321635 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318789    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 06:22:16.321635 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318791    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 06:22:16.321635 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318794    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 06:22:16.321635 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318796    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 06:22:16.321635 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318799    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 06:22:16.321635 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318801    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 06:22:16.321635 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318804    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 06:22:16.321635 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318806    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 06:22:16.321635 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318809    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 06:22:16.321635 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318813    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 06:22:16.321635 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318816    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 06:22:16.322111 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318819    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 06:22:16.322111 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318822    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 06:22:16.322111 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318825    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 06:22:16.322111 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318828    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 06:22:16.322111 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318832    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 06:22:16.322111 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318835    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 06:22:16.322111 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318840    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 06:22:16.322111 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318844    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 06:22:16.322111 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318847    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 06:22:16.322111 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318850    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 06:22:16.322111 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318853    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 06:22:16.322111 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318855    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 06:22:16.322111 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318858    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 06:22:16.322111 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318861    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 06:22:16.322111 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318864    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 06:22:16.322111 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318866    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 06:22:16.322111 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318869    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 06:22:16.322111 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318871    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 06:22:16.322111 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.318874    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 06:22:16.322593 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319258    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 06:22:16.322593 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319263    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 06:22:16.322593 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319266    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 06:22:16.322593 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319268    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 06:22:16.322593 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319271    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 06:22:16.322593 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319274    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 06:22:16.322593 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319276    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 06:22:16.322593 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319279    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 06:22:16.322593 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319281    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 06:22:16.322593 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319284    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 06:22:16.322593 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319288    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 06:22:16.322593 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319291    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 06:22:16.322593 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319293    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 06:22:16.322593 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319296    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 06:22:16.322593 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319299    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 06:22:16.322593 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319302    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 06:22:16.322593 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319304    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 06:22:16.322593 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319307    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 06:22:16.322593 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319310    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 06:22:16.322593 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319313    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 06:22:16.323080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319316    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 06:22:16.323080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319318    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 06:22:16.323080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319321    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 06:22:16.323080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319323    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 06:22:16.323080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319326    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 06:22:16.323080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319329    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 06:22:16.323080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319331    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 06:22:16.323080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319334    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 06:22:16.323080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319337    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 06:22:16.323080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319339    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 06:22:16.323080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319342    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 06:22:16.323080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319344    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 06:22:16.323080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319347    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 06:22:16.323080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319350    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 06:22:16.323080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319352    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 06:22:16.323080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319355    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 06:22:16.323080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319357    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 06:22:16.323080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319360    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 06:22:16.323080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319363    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 06:22:16.323080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319366    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 06:22:16.323584 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319373    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 06:22:16.323584 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319375    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 06:22:16.323584 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319378    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 06:22:16.323584 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319381    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 06:22:16.323584 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319383    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 06:22:16.323584 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319386    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 06:22:16.323584 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319388    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 06:22:16.323584 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319390    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 06:22:16.323584 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319393    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 06:22:16.323584 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319396    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 06:22:16.323584 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319398    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 06:22:16.323584 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319401    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 06:22:16.323584 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319406    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 06:22:16.323584 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319409    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 06:22:16.323584 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319411    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 06:22:16.323584 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319414    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 06:22:16.323584 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319418    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 06:22:16.323584 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319422    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 06:22:16.323584 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319425    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 06:22:16.324048 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319428    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 06:22:16.324048 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319433    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 06:22:16.324048 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319437    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 06:22:16.324048 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319440    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 06:22:16.324048 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319443    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 06:22:16.324048 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319446    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 06:22:16.324048 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319449    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 06:22:16.324048 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319452    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 06:22:16.324048 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319455    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 06:22:16.324048 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319459    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 06:22:16.324048 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319462    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 06:22:16.324048 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319465    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 06:22:16.324048 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319468    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 06:22:16.324048 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319471    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 06:22:16.324048 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319474    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 06:22:16.324048 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319476    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 06:22:16.324048 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319479    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 06:22:16.324048 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319481    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 06:22:16.324048 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319484    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 06:22:16.324627 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319487    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 06:22:16.324627 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319489    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 06:22:16.324627 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319492    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 06:22:16.324627 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319494    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 06:22:16.324627 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319497    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 06:22:16.324627 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319500    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 06:22:16.324627 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319502    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 06:22:16.324627 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319505    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 06:22:16.324627 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319508    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 06:22:16.324627 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319510    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 06:22:16.324627 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319513    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 06:22:16.324627 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319516    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 06:22:16.324627 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319518    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 06:22:16.324627 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319521    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 06:22:16.324627 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319523    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 06:22:16.324627 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319528    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 06:22:16.324627 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319533    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 06:22:16.324627 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319536    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 06:22:16.324627 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319539    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 06:22:16.324627 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319541    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.319544    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319614    2568 flags.go:64] FLAG: --address="0.0.0.0"
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319656    2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319664    2568 flags.go:64] FLAG: --anonymous-auth="true"
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319669    2568 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319674    2568 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319677    2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319682    2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319687    2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319690    2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319694    2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319697    2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319701    2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319704    2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319707    2568 flags.go:64] FLAG: --cgroup-root=""
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319710    2568 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319713    2568 flags.go:64] FLAG: --client-ca-file=""
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319716    2568 flags.go:64] FLAG: --cloud-provider="external"
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319719    2568 flags.go:64] FLAG: --cluster-dns="[]"
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319726    2568 flags.go:64] FLAG: --cluster-domain=""
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319735    2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319738    2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319741    2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 18 06:22:16.325125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319745    2568 flags.go:64] FLAG: --container-log-max-files="5"
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319749    2568 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319752    2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319755    2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319759    2568 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319764    2568 flags.go:64] FLAG: --contention-profiling="false"
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319767    2568 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319770    2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319773    2568 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319776    2568 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319781    2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319784    2568 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319787    2568 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319790    2568 flags.go:64] FLAG: --enable-load-reader="false"
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319793    2568 flags.go:64] FLAG: --enable-server="true"
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319796    2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319801    2568 flags.go:64] FLAG: --event-burst="100"
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319804    2568 flags.go:64] FLAG: --event-qps="50"
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319808    2568 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319811    2568 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319814    2568 flags.go:64] FLAG: --eviction-hard=""
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319817    2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319820    2568 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319824    2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319827    2568 flags.go:64] FLAG: --eviction-soft=""
Sep 18 06:22:16.325730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319830    2568 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319832    2568 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319835    2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319839    2568 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319841    2568 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319845    2568 flags.go:64] FLAG: --fail-swap-on="true"
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319848    2568 flags.go:64] FLAG: --feature-gates=""
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319852    2568 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319855    2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319858    2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319861    2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319864    2568 flags.go:64] FLAG: --healthz-port="10248"
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319869    2568 flags.go:64] FLAG: --help="false"
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319872    2568 flags.go:64] FLAG: --hostname-override="ip-10-0-128-30.ec2.internal"
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319875    2568 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319878    2568 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319881    2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319884    2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319888    2568 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319891    2568 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319894    2568 flags.go:64] FLAG: --image-service-endpoint=""
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319897    2568 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319900    2568 flags.go:64] FLAG: --kube-api-burst="100"
Sep 18 06:22:16.326354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319903    2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319906    2568 flags.go:64] FLAG: --kube-api-qps="50"
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319909    2568 flags.go:64] FLAG: --kube-reserved=""
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319912    2568 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319915    2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319918    2568 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319921    2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319924    2568 flags.go:64] FLAG: --lock-file=""
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319926    2568 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319929    2568 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319932    2568 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319938    2568 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319941    2568 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319944    2568 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319947    2568 flags.go:64] FLAG: --logging-format="text"
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319950    2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319954    2568 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319957    2568 flags.go:64] FLAG: --manifest-url=""
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319960    2568 flags.go:64] FLAG: --manifest-url-header=""
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319969    2568 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319973    2568 flags.go:64] FLAG: --max-open-files="1000000"
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319978    2568 flags.go:64] FLAG: --max-pods="110"
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319981    2568 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319984    2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319988    2568 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 18 06:22:16.327042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319991    2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319994    2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.319997    2568 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320000    2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320007    2568 flags.go:64] FLAG: --node-status-max-images="50"
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320010    2568 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320013    2568 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320018    2568 flags.go:64] FLAG: --pod-cidr=""
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320021    2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320026    2568 flags.go:64] FLAG: --pod-manifest-path=""
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320029    2568 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320033    2568 flags.go:64] FLAG: --pods-per-core="0"
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320036    2568 flags.go:64] FLAG: --port="10250"
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320039    2568 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320042    2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ebb85c0ebb4c2f8a"
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320045    2568 flags.go:64] FLAG: --qos-reserved=""
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320048    2568 flags.go:64] FLAG: --read-only-port="10255"
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320051    2568 flags.go:64] FLAG: --register-node="true"
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320054    2568 flags.go:64] FLAG: --register-with-taints=""
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320058    2568 flags.go:64] FLAG: --registry-burst="10"
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320061    2568 flags.go:64] FLAG: --registry-qps="5"
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320063    2568 flags.go:64] FLAG: --reserved-cpus=""
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320066    2568 flags.go:64] FLAG: --reserved-memory=""
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320070    2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320074    2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 18 06:22:16.327990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320077    2568 flags.go:64] FLAG: --rotate-certificates="false"
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320080    2568 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320083    2568 flags.go:64] FLAG: --runonce="false"
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320086    2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320090    2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320093    2568 flags.go:64] FLAG: --seccomp-default="false"
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320096    2568 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320099    2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320102    2568 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320106    2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320109    2568 flags.go:64] FLAG: --storage-driver-password="root"
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320112    2568 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320115    2568 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320118    2568 flags.go:64] FLAG: --storage-driver-user="root"
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320121    2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320124    2568 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320127    2568 flags.go:64] FLAG: --system-cgroups=""
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320130    2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320136    2568 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320139    2568 flags.go:64] FLAG: --tls-cert-file=""
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320142    2568 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320148    2568 flags.go:64] FLAG: --tls-min-version=""
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320151    2568 flags.go:64] FLAG: --tls-private-key-file=""
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320154    2568 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320157    2568 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 18 06:22:16.328701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320160    2568 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 18 06:22:16.329328 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320163    2568 flags.go:64] FLAG: --v="2"
Sep 18 06:22:16.329328 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320167    2568 flags.go:64] FLAG: --version="false"
Sep 18 06:22:16.329328 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320171    2568 flags.go:64] FLAG: --vmodule=""
Sep 18 06:22:16.329328 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320175    2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 18 06:22:16.329328 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.320178    2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 18 06:22:16.329328 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320317    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 06:22:16.329328 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320320    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 06:22:16.329328 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320324    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 06:22:16.329328 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320328    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 06:22:16.329328 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320332    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 06:22:16.329328 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320337    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 06:22:16.329328 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320340    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 06:22:16.329328 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320343    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 06:22:16.329328 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320346    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 06:22:16.329328 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320349    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 06:22:16.329328 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320352    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 06:22:16.329328 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320355    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 06:22:16.329328 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320358    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 06:22:16.329328 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320361    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 06:22:16.329328 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320363    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 06:22:16.329328 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320366    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 06:22:16.329859 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320368    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 06:22:16.329859 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320371    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 06:22:16.329859 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320374    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 06:22:16.329859 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320377    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 06:22:16.329859 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320379    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 06:22:16.329859 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320382    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 06:22:16.329859 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320385    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 06:22:16.329859 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320387    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 06:22:16.329859 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320390    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 06:22:16.329859 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320393    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 06:22:16.329859 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320396    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 06:22:16.329859 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320398    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 06:22:16.329859 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320401    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 06:22:16.329859 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320403    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 06:22:16.329859 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320406    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 06:22:16.329859 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320409    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 06:22:16.329859 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320411    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 06:22:16.329859 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320427    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 06:22:16.329859 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320430    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 06:22:16.329859 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320434    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 06:22:16.330370 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320437    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 06:22:16.330370 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320441    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 06:22:16.330370 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320444    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 06:22:16.330370 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320446    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 06:22:16.330370 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320449    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 06:22:16.330370 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320451    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 06:22:16.330370 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320454    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 06:22:16.330370 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320457    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 06:22:16.330370 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320459    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 06:22:16.330370 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320462    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 06:22:16.330370 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320465    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 06:22:16.330370 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320468    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 06:22:16.330370 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320470    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 06:22:16.330370 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320473    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 06:22:16.330370 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320476    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 06:22:16.330370 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320478    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 06:22:16.330370 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320481    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 06:22:16.330370 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320484    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 06:22:16.330370 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320486    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 06:22:16.330370 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320489    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 06:22:16.330873 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320491    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 06:22:16.330873 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320494    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 06:22:16.330873 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320497    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 06:22:16.330873 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320499    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 06:22:16.330873 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320502    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 06:22:16.330873 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320510    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 06:22:16.330873 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320514    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 06:22:16.330873 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320517    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 06:22:16.330873 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320521    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 06:22:16.330873 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320524    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 06:22:16.330873 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320533    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 06:22:16.330873 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320539    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 06:22:16.330873 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320543    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 06:22:16.330873 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320547    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 06:22:16.330873 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320551    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 06:22:16.330873 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320554    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 06:22:16.330873 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320557    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 06:22:16.330873 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320559    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 06:22:16.330873 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320562    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 06:22:16.331371 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320565    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 06:22:16.331371 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320568    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 06:22:16.331371 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320570    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 06:22:16.331371 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320573    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 06:22:16.331371 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320575    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 06:22:16.331371 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320578    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 06:22:16.331371 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320581    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 06:22:16.331371 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320583    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 06:22:16.331371 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320586    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 06:22:16.331371 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320589    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 06:22:16.331371 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320592    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 06:22:16.331371 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320594    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 06:22:16.331371 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320597    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 06:22:16.331371 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320600    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 06:22:16.331371 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320602    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 06:22:16.331371 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320605    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 06:22:16.331371 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320608    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 06:22:16.331371 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320610    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 06:22:16.331371 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320613    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 06:22:16.331371 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320615    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 06:22:16.331863 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320619    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 06:22:16.331863 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320623    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 06:22:16.331863 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320626    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 06:22:16.331863 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.320629    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 06:22:16.331863 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.321190    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 06:22:16.331863 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.329496    2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 18 06:22:16.331863 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.329511    2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 18 06:22:16.331863 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.329534    2568 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 18 06:22:16.331863 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.329540    2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 18 06:22:16.331863 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329596    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 06:22:16.331863 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329602    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 06:22:16.331863 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329605    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 06:22:16.331863 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329609    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 06:22:16.331863 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329612    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 06:22:16.332228 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329615    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 06:22:16.332228 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329618    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 06:22:16.332228 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329620    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 06:22:16.332228 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329623    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 06:22:16.332228 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329626    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 06:22:16.332228 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329629    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 06:22:16.332228 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329631    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 06:22:16.332228 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329634    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 06:22:16.332228 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329636    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 06:22:16.332228 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329639    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 06:22:16.332228 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329642    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 06:22:16.332228 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329644    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 06:22:16.332228 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329647    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 06:22:16.332228 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329650    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 06:22:16.332228 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329652    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 06:22:16.332228 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329655    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 06:22:16.332228 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329658    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 06:22:16.332228 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329660    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 06:22:16.332228 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329666    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 06:22:16.332228 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329669    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 06:22:16.332784 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329671    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 06:22:16.332784 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329674    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 06:22:16.332784 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329677    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 06:22:16.332784 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329680    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 06:22:16.332784 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329682    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 06:22:16.332784 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329685    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 06:22:16.332784 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329688    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 06:22:16.332784 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329691    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 06:22:16.332784 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329693    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 06:22:16.332784 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329696    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 06:22:16.332784 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329699    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 06:22:16.332784 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329701    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 06:22:16.332784 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329704    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 06:22:16.332784 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329706    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 06:22:16.332784 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329709    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 06:22:16.332784 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329712    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 06:22:16.332784 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329714    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 06:22:16.332784 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329719    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 06:22:16.332784 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329722    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 06:22:16.332784 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329725    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 06:22:16.333296 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329728    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 06:22:16.333296 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329731    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 06:22:16.333296 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329734    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 06:22:16.333296 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329737    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 06:22:16.333296 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329740    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 06:22:16.333296 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329742    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 06:22:16.333296 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329745    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 06:22:16.333296 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329748    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 06:22:16.333296 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329751    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 06:22:16.333296 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329753    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 06:22:16.333296 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329756    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 06:22:16.333296 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329758    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 06:22:16.333296 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329761    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 06:22:16.333296 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329764    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 06:22:16.333296 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329766    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 06:22:16.333296 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329769    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 06:22:16.333296 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329772    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 06:22:16.333296 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329776    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 06:22:16.333296 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329780    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 06:22:16.333296 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329783    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 06:22:16.334080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329786    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 06:22:16.334080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329788    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 06:22:16.334080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329791    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 06:22:16.334080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329793    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 06:22:16.334080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329796    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 06:22:16.334080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329798    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 06:22:16.334080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329801    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 06:22:16.334080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329804    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 06:22:16.334080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329808    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 06:22:16.334080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329812    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 06:22:16.334080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329815    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 06:22:16.334080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329818    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 06:22:16.334080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329821    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 06:22:16.334080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329823    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 06:22:16.334080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329827    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 06:22:16.334080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329829    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 06:22:16.334080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329832    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 06:22:16.334080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329834    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 06:22:16.334080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329837    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 06:22:16.334080 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329840    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 06:22:16.334634 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329842    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 06:22:16.334634 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329845    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 06:22:16.334634 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329847    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 06:22:16.334634 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329850    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 06:22:16.334634 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329853    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 06:22:16.334634 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329855    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 06:22:16.334634 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329858    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 06:22:16.334634 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329861    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 06:22:16.334634 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329863    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 06:22:16.334634 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329866    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 06:22:16.334634 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329869    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 06:22:16.334634 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329872    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 06:22:16.334634 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329875    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 06:22:16.334634 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.329878    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 06:22:16.334634 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.329883    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 06:22:16.335133 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330001    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 06:22:16.335133 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330006    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 06:22:16.335133 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330009    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 06:22:16.335133 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330012    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 06:22:16.335133 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330015    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 06:22:16.335133 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330018    2568 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 06:22:16.335133 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330021    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 06:22:16.335133 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330024    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 06:22:16.335133 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330027    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 06:22:16.335133 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330030    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 06:22:16.335133 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330033    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 06:22:16.335133 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330035    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 06:22:16.335133 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330039    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 06:22:16.335133 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330042    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 06:22:16.335133 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330044    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 06:22:16.335133 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330048    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 06:22:16.335133 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330051    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 06:22:16.335133 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330053    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 06:22:16.335133 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330056    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 06:22:16.335133 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330059    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 06:22:16.335731 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330062    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 06:22:16.335731 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330065    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 06:22:16.335731 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330069    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 06:22:16.335731 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330071    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 06:22:16.335731 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330074    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 06:22:16.335731 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330077    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 06:22:16.335731 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330079    2568 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 06:22:16.335731 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330084    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 06:22:16.335731 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330088    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 06:22:16.335731 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330092    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 06:22:16.335731 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330095    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 06:22:16.335731 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330098    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 06:22:16.335731 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330100    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 06:22:16.335731 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330103    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 06:22:16.335731 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330106    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 06:22:16.335731 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330108    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 06:22:16.335731 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330111    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 06:22:16.335731 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330114    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 06:22:16.335731 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330116    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 06:22:16.336291 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330119    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 06:22:16.336291 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330121    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 06:22:16.336291 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330124    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 06:22:16.336291 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330127    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 06:22:16.336291 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330129    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 06:22:16.336291 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330132    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 06:22:16.336291 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330135    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 06:22:16.336291 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330137    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 06:22:16.336291 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330142    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 06:22:16.336291 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330145    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 06:22:16.336291 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330147    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 06:22:16.336291 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330151    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 06:22:16.336291 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330153    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 06:22:16.336291 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330156    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 06:22:16.336291 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330159    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 06:22:16.336291 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330161    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 06:22:16.336291 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330164    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 06:22:16.336291 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330166    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 06:22:16.336291 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330169    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 06:22:16.336291 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330172    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 06:22:16.336848 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330174    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 06:22:16.336848 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330177    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 06:22:16.336848 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330180    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 06:22:16.336848 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330182    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 06:22:16.336848 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330185    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 06:22:16.336848 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330188    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 06:22:16.336848 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330191    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 06:22:16.336848 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330193    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 06:22:16.336848 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330196    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 06:22:16.336848 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330198    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 06:22:16.336848 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330201    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 06:22:16.336848 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330204    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 06:22:16.336848 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330206    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 06:22:16.336848 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330209    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 06:22:16.336848 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330211    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 06:22:16.336848 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330225    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 06:22:16.336848 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330228    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 06:22:16.336848 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330231    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 06:22:16.336848 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330234    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 06:22:16.336848 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330237    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 06:22:16.337417 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330240    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 06:22:16.337417 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330243    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 06:22:16.337417 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330246    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 06:22:16.337417 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330248    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 06:22:16.337417 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330251    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 06:22:16.337417 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330253    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 06:22:16.337417 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330256    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 06:22:16.337417 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330260    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 06:22:16.337417 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330263    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 06:22:16.337417 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330266    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 06:22:16.337417 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330269    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 06:22:16.337417 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330272    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 06:22:16.337417 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330275    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 06:22:16.337417 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330277    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 06:22:16.337417 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330280    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 06:22:16.337417 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330283    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 06:22:16.337417 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330286    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 06:22:16.337417 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330288    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 06:22:16.337417 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330291    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 06:22:16.338014 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:16.330294    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 06:22:16.338014 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.330299    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 06:22:16.338014 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.330403    2568 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 18 06:22:16.338014 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.332436    2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 18 06:22:16.338014 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.333290    2568 server.go:1019] "Starting client certificate rotation"
Sep 18 06:22:16.338014 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.333404    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 06:22:16.338014 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.333454    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 06:22:16.338014 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.336026    2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 06:22:16.338014 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.337176    2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 06:22:16.348603 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.348575    2568 log.go:25] "Validated CRI v1 runtime API"
Sep 18 06:22:16.350303 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.350285    2568 log.go:25] "Validated CRI v1 image API"
Sep 18 06:22:16.351056 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.351044    2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 18 06:22:16.353849 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.353823    2568 fs.go:135] Filesystem UUIDs: map[2828c5b9-2fdd-4a26-b3a1-a77700a580c4:/dev/nvme0n1p4 352c13a5-2074-4596-8d62-711e1fe39bf7:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Sep 18 06:22:16.353917 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.353846    2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 18 06:22:16.358707 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.358400    2568 manager.go:216] Machine: {Timestamp:2026-09-18 06:22:16.357537707 +0000 UTC m=+2.039065444 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103150 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec213ca0cdf327ee264a44e345ab97c0 SystemUUID:ec213ca0-cdf3-27ee-264a-44e345ab97c0 BootID:aea17226-9419-497e-aed2-880fa4353c37 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f1:9f:30:e7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f1:9f:30:e7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:84:45:8e:64:3a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 18 06:22:16.358707 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.358634    2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 18 06:22:16.358860 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.358746    2568 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 18 06:22:16.360371 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.360335    2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 18 06:22:16.360530 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.360386    2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-30.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 18 06:22:16.361526 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.361514    2568 topology_manager.go:138] "Creating topology manager with none policy"
Sep 18 06:22:16.361574 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.361532    2568 container_manager_linux.go:309] "Creating device plugin manager"
Sep 18 06:22:16.361574 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.361542    2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 06:22:16.361574 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.361554    2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 06:22:16.361659 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.361626    2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 18 06:22:16.364271 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.364259    2568 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 06:22:16.364385 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.364376    2568 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 18 06:22:16.364428 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.364409    2568 kubelet.go:486] "Attempting to sync node with API server"
Sep 18 06:22:16.364428 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.364419    2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 18 06:22:16.364524 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.364434    2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 18 06:22:16.364524 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.364444    2568 kubelet.go:398] "Adding apiserver pod source"
Sep 18 06:22:16.364524 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.364453    2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 18 06:22:16.364795 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.364782    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 06:22:16.364830 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.364806    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 06:22:16.366728 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.366704    2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 18 06:22:16.368933 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.368915    2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 18 06:22:16.369024 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.368954    2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 18 06:22:16.369844 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.369831    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 18 06:22:16.369920 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.369851    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 18 06:22:16.369920 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.369859    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 18 06:22:16.369920 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.369867    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 18 06:22:16.369920 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.369881    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 18 06:22:16.369920 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.369891    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 18 06:22:16.369920 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.369912    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 18 06:22:16.369920 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.369921    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 18 06:22:16.370165 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.369931    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 18 06:22:16.370165 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.369940    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 18 06:22:16.370165 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.369953    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 18 06:22:16.370165 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.369961    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 18 06:22:16.370165 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.369995    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 18 06:22:16.370165 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.370004    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 18 06:22:16.371207 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.371189    2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 06:22:16.374061 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.374038    2568 server.go:1268] "Started kubelet"
Sep 18 06:22:16.374174 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.374111    2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 18 06:22:16.374249 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.374178    2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 18 06:22:16.374249 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.374197    2568 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 18 06:22:16.374958 ip-10-0-128-30 systemd[1]: Started Kubernetes Kubelet.
Sep 18 06:22:16.375523 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.375161    2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 18 06:22:16.375523 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.375376    2568 server.go:310] "Adding debug handlers to kubelet server"
Sep 18 06:22:16.375637 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.375604    2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-30.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 18 06:22:16.376147 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:16.376049    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-30.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 18 06:22:16.376147 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:16.376041    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 18 06:22:16.378403 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:16.378386    2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 18 06:22:16.380906 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.380889    2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 18 06:22:16.380987 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.380909    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 18 06:22:16.381164 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:16.381134    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 18 06:22:16.381457 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.381441    2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 18 06:22:16.381530 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.381461    2568 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 18 06:22:16.381530 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.381474    2568 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 18 06:22:16.381687 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.381670    2568 factory.go:55] Registering systemd factory
Sep 18 06:22:16.381752 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.381691    2568 factory.go:223] Registration of the systemd container factory successfully
Sep 18 06:22:16.381944 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.381931    2568 factory.go:153] Registering CRI-O factory
Sep 18 06:22:16.381995 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.381946    2568 factory.go:223] Registration of the crio container factory successfully
Sep 18 06:22:16.382026 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.381994    2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 18 06:22:16.382026 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.382012    2568 factory.go:103] Registering Raw factory
Sep 18 06:22:16.382026 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.382022    2568 manager.go:1196] Started watching for new ooms in manager
Sep 18 06:22:16.382307 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.382294    2568 reconstruct.go:97] "Volume reconstruction finished"
Sep 18 06:22:16.382339 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.382309    2568 reconciler.go:29] "Reconciler: start to sync state"
Sep 18 06:22:16.382339 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.382334    2568 manager.go:349] Starting recovery of all containers
Sep 18 06:22:16.383578 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:16.383555    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 18 06:22:16.383670 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:16.383653    2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-30.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 18 06:22:16.405180 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.405157    2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-d7l64"
Sep 18 06:22:16.406107 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.406083    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-d7l64"
Sep 18 06:22:16.411423 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.411405    2568 manager.go:354] Recovery completed
Sep 18 06:22:16.415343 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.415329    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 06:22:16.417480 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.417465    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 06:22:16.417566 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.417496    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 06:22:16.417566 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.417510    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientPID"
Sep 18 06:22:16.418045 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.418030    2568 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 18 06:22:16.418045 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.418041    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 06:22:16.418161 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.418042    2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 18 06:22:16.418161 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.418109    2568 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 06:22:16.418699 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.418675    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 06:22:16.418699 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.418702    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 06:22:16.418800 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.418717    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientPID"
Sep 18 06:22:16.419319 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.419308    2568 policy_none.go:49] "None policy: Start"
Sep 18 06:22:16.419365 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.419324    2568 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 18 06:22:16.419365 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.419335    2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 18 06:22:16.420183 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.420174    2568 policy_none.go:47] "Start"
Sep 18 06:22:16.459782 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.459764    2568 manager.go:337] "Starting Device Plugin manager"
Sep 18 06:22:16.473145 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:16.459807    2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 18 06:22:16.473145 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.459822    2568 server.go:87] "Starting device plugin registration server"
Sep 18 06:22:16.473145 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.460054    2568 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 18 06:22:16.473145 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.460072    2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 18 06:22:16.473145 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.460146    2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 18 06:22:16.473145 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.460264    2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 18 06:22:16.473145 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.460270    2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 18 06:22:16.473145 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:16.460670    2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 18 06:22:16.473145 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:16.460706    2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 18 06:22:16.535187 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.535153    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 18 06:22:16.536257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.536244    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 18 06:22:16.536335 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.536288    2568 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 18 06:22:16.536335 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.536309    2568 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 18 06:22:16.536412 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:16.536345    2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 18 06:22:16.538668 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.538650    2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 06:22:16.560402 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.560386    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 06:22:16.561213 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.561197    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 06:22:16.561307 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.561244    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 06:22:16.561307 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.561257    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientPID"
Sep 18 06:22:16.561307 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.561281    2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-30.ec2.internal"
Sep 18 06:22:16.568049 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.568004    2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-30.ec2.internal"
Sep 18 06:22:16.568049 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:16.568025    2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-30.ec2.internal\": node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 18 06:22:16.583482 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:16.583462    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 18 06:22:16.637269 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.637245    2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal"]
Sep 18 06:22:16.637342 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.637323    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 06:22:16.638163 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.638149    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 06:22:16.638243 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.638174    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 06:22:16.638243 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.638184    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientPID"
Sep 18 06:22:16.639337 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.639326    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 06:22:16.639445 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.639432    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal"
Sep 18 06:22:16.639486 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.639472    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 06:22:16.640012 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.639998    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 06:22:16.640104 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.640022    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 06:22:16.640104 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.640031    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientPID"
Sep 18 06:22:16.640470 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.640443    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 06:22:16.640548 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.640480    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 06:22:16.640548 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.640492    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientPID"
Sep 18 06:22:16.641690 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.641675    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal"
Sep 18 06:22:16.641760 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.641699    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 06:22:16.642384 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.642371    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 06:22:16.642459 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.642400    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 06:22:16.642459 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.642415    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientPID"
Sep 18 06:22:16.667828 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:16.667813    2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-30.ec2.internal\" not found" node="ip-10-0-128-30.ec2.internal"
Sep 18 06:22:16.672424 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:16.672409    2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-30.ec2.internal\" not found" node="ip-10-0-128-30.ec2.internal"
Sep 18 06:22:16.684041 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:16.684023    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 18 06:22:16.684127 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.684112    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4a8bd5eb6bfe7e7a25b8f8115ad4a547-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal\" (UID: \"4a8bd5eb6bfe7e7a25b8f8115ad4a547\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal"
Sep 18 06:22:16.684167 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.684141    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a8bd5eb6bfe7e7a25b8f8115ad4a547-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal\" (UID: \"4a8bd5eb6bfe7e7a25b8f8115ad4a547\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal"
Sep 18 06:22:16.684167 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.684158    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aab83480badcc6c24454042cf62b9928-config\") pod \"kube-apiserver-proxy-ip-10-0-128-30.ec2.internal\" (UID: \"aab83480badcc6c24454042cf62b9928\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal"
Sep 18 06:22:16.784783 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:16.784754    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 18 06:22:16.784871 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.784810    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4a8bd5eb6bfe7e7a25b8f8115ad4a547-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal\" (UID: \"4a8bd5eb6bfe7e7a25b8f8115ad4a547\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal"
Sep 18 06:22:16.784871 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.784834    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4a8bd5eb6bfe7e7a25b8f8115ad4a547-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal\" (UID: \"4a8bd5eb6bfe7e7a25b8f8115ad4a547\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal"
Sep 18 06:22:16.784936 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.784878    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a8bd5eb6bfe7e7a25b8f8115ad4a547-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal\" (UID: \"4a8bd5eb6bfe7e7a25b8f8115ad4a547\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal"
Sep 18 06:22:16.784936 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.784903    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aab83480badcc6c24454042cf62b9928-config\") pod \"kube-apiserver-proxy-ip-10-0-128-30.ec2.internal\" (UID: \"aab83480badcc6c24454042cf62b9928\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal"
Sep 18 06:22:16.784936 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.784928    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/aab83480badcc6c24454042cf62b9928-config\") pod \"kube-apiserver-proxy-ip-10-0-128-30.ec2.internal\" (UID: \"aab83480badcc6c24454042cf62b9928\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal"
Sep 18 06:22:16.785025 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.784961    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a8bd5eb6bfe7e7a25b8f8115ad4a547-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal\" (UID: \"4a8bd5eb6bfe7e7a25b8f8115ad4a547\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal"
Sep 18 06:22:16.885422 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:16.885345    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 18 06:22:16.972579 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.972555    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal"
Sep 18 06:22:16.975777 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:16.975751    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal"
Sep 18 06:22:16.986064 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:16.986045    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 18 06:22:17.086567 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:17.086385    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 18 06:22:17.186853 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:17.186768    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 18 06:22:17.287158 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:17.287130    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 18 06:22:17.333311 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:17.333282    2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 18 06:22:17.333762 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:17.333466    2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 06:22:17.387276 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:17.387250    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 18 06:22:17.411361 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:17.411329    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-17 06:17:16 +0000 UTC" deadline="2028-04-14 07:50:20.242807708 +0000 UTC"
Sep 18 06:22:17.411361 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:17.411354    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13777h28m2.831456143s"
Sep 18 06:22:17.442713 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:17.442657    2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 06:22:17.487752 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:17.487724    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 18 06:22:17.556567 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:17.556544    2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 06:22:17.582190 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:17.582163    2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal"
Sep 18 06:22:17.582925 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:17.582908    2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 06:22:17.593173 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:17.593155    2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 06:22:17.594019 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:17.594005    2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal"
Sep 18 06:22:17.601674 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:17.601643    2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 06:22:17.703864 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:17.703802    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaab83480badcc6c24454042cf62b9928.slice/crio-9f82f281f4479143dd582ca77ad5f779f339654a60826c31dd3d20957074e373 WatchSource:0}: Error finding container 9f82f281f4479143dd582ca77ad5f779f339654a60826c31dd3d20957074e373: Status 404 returned error can't find the container with id 9f82f281f4479143dd582ca77ad5f779f339654a60826c31dd3d20957074e373
Sep 18 06:22:17.704259 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:17.704240    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a8bd5eb6bfe7e7a25b8f8115ad4a547.slice/crio-8682d3941337917cb49844804c00a76806c8ecc1efb804f04d4f8fe26fd1bf75 WatchSource:0}: Error finding container 8682d3941337917cb49844804c00a76806c8ecc1efb804f04d4f8fe26fd1bf75: Status 404 returned error can't find the container with id 8682d3941337917cb49844804c00a76806c8ecc1efb804f04d4f8fe26fd1bf75
Sep 18 06:22:17.710032 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:17.710018    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 06:22:18.366627 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.366591    2568 apiserver.go:52] "Watching apiserver"
Sep 18 06:22:18.375109 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.375084    2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 18 06:22:18.375610 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.375581    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-2ffp7","kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal","openshift-multus/multus-additional-cni-plugins-z26mm","openshift-multus/multus-nssrf","openshift-network-diagnostics/network-check-target-98gq6","openshift-network-operator/iptables-alerter-bgkn4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9","openshift-cluster-node-tuning-operator/tuned-cqb8p","openshift-dns/node-resolver-nrnz8","openshift-image-registry/node-ca-6lmr2","openshift-multus/network-metrics-daemon-k6z9c","openshift-ovn-kubernetes/ovnkube-node-l6kp8","kube-system/global-pull-secret-syncer-n8fkx"]
Sep 18 06:22:18.377370 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.377342    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bgkn4"
Sep 18 06:22:18.379091 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.378860    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.379661 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.379576    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2mkml\""
Sep 18 06:22:18.380574 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.379906    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 18 06:22:18.380574 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.379930    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 18 06:22:18.380574 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.380138    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 18 06:22:18.380574 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.380358    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:18.380574 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:18.380420    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8fkx" podUID="1a0016b5-3d41-4c0c-afd6-bf60b5e69237"
Sep 18 06:22:18.380989 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.380911    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 18 06:22:18.380989 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.380981    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gc69j\""
Sep 18 06:22:18.381105 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.381085    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 18 06:22:18.381168 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.381138    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 18 06:22:18.381232 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.381165    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 18 06:22:18.381542 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.381524    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.382148 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.381907    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 18 06:22:18.382148 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.382029    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 18 06:22:18.382148 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.382104    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 18 06:22:18.383944 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.383857    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.384044 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.384003    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 18 06:22:18.384104 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.384045    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 18 06:22:18.384287 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.384270    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xbvln\""
Sep 18 06:22:18.384575 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.384380    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 18 06:22:18.384575 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.384406    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 18 06:22:18.384575 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.384480    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 18 06:22:18.385549 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.385524    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:18.385647 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:18.385587    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98gq6" podUID="5b9b648c-4c7d-4f02-a6b3-c8e719f972b4"
Sep 18 06:22:18.386371 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.386353    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nrw2l\""
Sep 18 06:22:18.386461 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.386374    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 18 06:22:18.388005 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.387791    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.389553 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.389025    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2ffp7"
Sep 18 06:22:18.389951 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.389931    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 18 06:22:18.390115 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.390099    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jm5hn\""
Sep 18 06:22:18.390542 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.390359    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 18 06:22:18.392257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.390716    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.392257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.390825    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk9h7\" (UniqueName: \"kubernetes.io/projected/b76b1c21-d297-4f27-bc91-208669dc2890-kube-api-access-rk9h7\") pod \"iptables-alerter-bgkn4\" (UID: \"b76b1c21-d297-4f27-bc91-208669dc2890\") " pod="openshift-network-operator/iptables-alerter-bgkn4"
Sep 18 06:22:18.392257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.390856    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-host-kubelet\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.392257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.390880    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-systemd-units\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.392257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.390905    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-run-openvswitch\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.392257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.390931    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-run-ovn\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.392257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.390954    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-host-run-netns\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.392257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.390977    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfk9t\" (UniqueName: \"kubernetes.io/projected/e700d4ba-66b4-429a-bae1-74b61fc48253-kube-api-access-vfk9t\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.392257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391003    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-host-slash\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.392257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391028    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.392257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391055    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-kubelet-config\") pod \"global-pull-secret-syncer-n8fkx\" (UID: \"1a0016b5-3d41-4c0c-afd6-bf60b5e69237\") " pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:18.392257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391080    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkvh6\" (UniqueName: \"kubernetes.io/projected/688032c8-c1d8-4121-8ee0-c224615cc527-kube-api-access-kkvh6\") pod \"multus-additional-cni-plugins-z26mm\" (UID: \"688032c8-c1d8-4121-8ee0-c224615cc527\") " pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.392257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391121    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-cnibin\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.392257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391143    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e700d4ba-66b4-429a-bae1-74b61fc48253-multus-daemon-config\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.392257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391176    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-log-socket\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.392257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391202    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret\") pod \"global-pull-secret-syncer-n8fkx\" (UID: \"1a0016b5-3d41-4c0c-afd6-bf60b5e69237\") " pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:18.392257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391243    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/688032c8-c1d8-4121-8ee0-c224615cc527-system-cni-dir\") pod \"multus-additional-cni-plugins-z26mm\" (UID: \"688032c8-c1d8-4121-8ee0-c224615cc527\") " pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.393094 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391268    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/688032c8-c1d8-4121-8ee0-c224615cc527-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z26mm\" (UID: \"688032c8-c1d8-4121-8ee0-c224615cc527\") " pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.393094 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391293    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-host-run-multus-certs\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.393094 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391315    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b76b1c21-d297-4f27-bc91-208669dc2890-host-slash\") pod \"iptables-alerter-bgkn4\" (UID: \"b76b1c21-d297-4f27-bc91-208669dc2890\") " pod="openshift-network-operator/iptables-alerter-bgkn4"
Sep 18 06:22:18.393094 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391345    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-host-run-ovn-kubernetes\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.393094 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391381    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/688032c8-c1d8-4121-8ee0-c224615cc527-cni-binary-copy\") pod \"multus-additional-cni-plugins-z26mm\" (UID: \"688032c8-c1d8-4121-8ee0-c224615cc527\") " pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.393094 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391404    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e700d4ba-66b4-429a-bae1-74b61fc48253-cni-binary-copy\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.393094 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391432    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-host-run-k8s-cni-cncf-io\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.393094 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391467    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-host-var-lib-cni-bin\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.393094 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391489    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-multus-conf-dir\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.393094 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391511    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-etc-kubernetes\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.393094 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391534    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-host-cni-bin\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.393094 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391562    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-env-overrides\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.393094 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391586    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-ovn-node-metrics-cert\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.393094 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391611    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b76b1c21-d297-4f27-bc91-208669dc2890-iptables-alerter-script\") pod \"iptables-alerter-bgkn4\" (UID: \"b76b1c21-d297-4f27-bc91-208669dc2890\") " pod="openshift-network-operator/iptables-alerter-bgkn4"
Sep 18 06:22:18.393094 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391635    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-node-log\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.393094 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391683    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/688032c8-c1d8-4121-8ee0-c224615cc527-cnibin\") pod \"multus-additional-cni-plugins-z26mm\" (UID: \"688032c8-c1d8-4121-8ee0-c224615cc527\") " pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.393716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391710    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-system-cni-dir\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.393716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391731    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-host-run-netns\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.393716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391745    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 18 06:22:18.393716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391756    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-run-systemd\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.393716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391782    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-ovnkube-config\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.393716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391806    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ghww\" (UniqueName: \"kubernetes.io/projected/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-kube-api-access-7ghww\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.393716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391829    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/688032c8-c1d8-4121-8ee0-c224615cc527-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z26mm\" (UID: \"688032c8-c1d8-4121-8ee0-c224615cc527\") " pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.393716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391850    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-multus-socket-dir-parent\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.393716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391875    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-host-var-lib-kubelet\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.393716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391895    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-hostroot\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.393716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391917    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-var-lib-openvswitch\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.393716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391939    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-etc-openvswitch\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.393716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391966    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-ovnkube-script-lib\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.393716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.391991    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-multus-cni-dir\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.393716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.392019    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-host-var-lib-cni-multus\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.393716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.392031    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 18 06:22:18.393716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.392051    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-host-cni-netd\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.393716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.392076    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-dbus\") pod \"global-pull-secret-syncer-n8fkx\" (UID: \"1a0016b5-3d41-4c0c-afd6-bf60b5e69237\") " pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:18.394354 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.392099    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/688032c8-c1d8-4121-8ee0-c224615cc527-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z26mm\" (UID: \"688032c8-c1d8-4121-8ee0-c224615cc527\") " pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.398869 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.392245    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cqdmx\""
Sep 18 06:22:18.398869 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.398347    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 18 06:22:18.398869 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.398431    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 18 06:22:18.398869 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.398578    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xbl5k\""
Sep 18 06:22:18.399101 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.398867    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 18 06:22:18.400163 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.398966    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nrnz8"
Sep 18 06:22:18.400778 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.400559    2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 06:22:18.401357 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.400942    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6lmr2"
Sep 18 06:22:18.401357 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.401231    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-t779k\""
Sep 18 06:22:18.402013 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.401835    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 18 06:22:18.402013 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.401866    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 18 06:22:18.403129 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.403105    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 18 06:22:18.403657 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.403601    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-w5zhk\""
Sep 18 06:22:18.403789 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.403771    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 18 06:22:18.403929 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.403893    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:18.404024 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.403946    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 18 06:22:18.404024 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:18.403988    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6z9c" podUID="bcf277d6-51cd-4bc8-ba3f-e6fcf272d363"
Sep 18 06:22:18.413475 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.413451    2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qlt4v"
Sep 18 06:22:18.423167 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.423149    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qlt4v"
Sep 18 06:22:18.482806 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.482789    2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 18 06:22:18.493136 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493108    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-host-var-lib-kubelet\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.493284 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493150    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-ovn-node-metrics-cert\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.493284 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493179    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-node-log\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.493284 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493205    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/688032c8-c1d8-4121-8ee0-c224615cc527-cnibin\") pod \"multus-additional-cni-plugins-z26mm\" (UID: \"688032c8-c1d8-4121-8ee0-c224615cc527\") " pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.493284 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493238    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-host-var-lib-kubelet\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.493445 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493245    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1ad3cc85-481a-4042-bc66-5214457747e0-sys-fs\") pod \"aws-ebs-csi-driver-node-ngvf9\" (UID: \"1ad3cc85-481a-4042-bc66-5214457747e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.493445 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493383    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqppv\" (UniqueName: \"kubernetes.io/projected/1ad3cc85-481a-4042-bc66-5214457747e0-kube-api-access-nqppv\") pod \"aws-ebs-csi-driver-node-ngvf9\" (UID: \"1ad3cc85-481a-4042-bc66-5214457747e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.493445 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493408    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/612baaf8-db1e-44ca-993e-836959f23b2d-serviceca\") pod \"node-ca-6lmr2\" (UID: \"612baaf8-db1e-44ca-993e-836959f23b2d\") " pod="openshift-image-registry/node-ca-6lmr2"
Sep 18 06:22:18.493445 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493435    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-run-systemd\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.493579 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493459    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-ovnkube-config\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.493579 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493484    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ghww\" (UniqueName: \"kubernetes.io/projected/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-kube-api-access-7ghww\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.493579 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493507    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-multus-socket-dir-parent\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.493579 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493532    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-etc-sysctl-d\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.493579 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493554    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-sys\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.493832 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493579    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1ad3cc85-481a-4042-bc66-5214457747e0-device-dir\") pod \"aws-ebs-csi-driver-node-ngvf9\" (UID: \"1ad3cc85-481a-4042-bc66-5214457747e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.493832 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493606    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d1f8c4d2-31e1-4cca-b763-dc559fa12937-konnectivity-ca\") pod \"konnectivity-agent-2ffp7\" (UID: \"d1f8c4d2-31e1-4cca-b763-dc559fa12937\") " pod="kube-system/konnectivity-agent-2ffp7"
Sep 18 06:22:18.493832 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493630    2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 18 06:22:18.493832 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493631    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-host-var-lib-cni-multus\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.493832 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493670    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ba599bd-531c-4d30-829f-b5cde2f02132-tmp\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.493832 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493706    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26tg7\" (UniqueName: \"kubernetes.io/projected/8ba599bd-531c-4d30-829f-b5cde2f02132-kube-api-access-26tg7\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.493832 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493737    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1ad3cc85-481a-4042-bc66-5214457747e0-etc-selinux\") pod \"aws-ebs-csi-driver-node-ngvf9\" (UID: \"1ad3cc85-481a-4042-bc66-5214457747e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.493832 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493760    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ead9d685-da05-4132-b36c-4887546472f2-hosts-file\") pod \"node-resolver-nrnz8\" (UID: \"ead9d685-da05-4132-b36c-4887546472f2\") " pod="openshift-dns/node-resolver-nrnz8"
Sep 18 06:22:18.493832 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493784    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-host-cni-netd\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.493832 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493813    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-dbus\") pod \"global-pull-secret-syncer-n8fkx\" (UID: \"1a0016b5-3d41-4c0c-afd6-bf60b5e69237\") " pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:18.493832 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493836    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8ba599bd-531c-4d30-829f-b5cde2f02132-etc-tuned\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.494393 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493864    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk9h7\" (UniqueName: \"kubernetes.io/projected/b76b1c21-d297-4f27-bc91-208669dc2890-kube-api-access-rk9h7\") pod \"iptables-alerter-bgkn4\" (UID: \"b76b1c21-d297-4f27-bc91-208669dc2890\") " pod="openshift-network-operator/iptables-alerter-bgkn4"
Sep 18 06:22:18.494393 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493914    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-host-var-lib-cni-multus\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.494393 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493918    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-host-kubelet\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.494393 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493963    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-host-kubelet\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.494393 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493964    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-run-openvswitch\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.494393 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.493994    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-run-openvswitch\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.494393 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494172    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-ovnkube-config\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.494393 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494191    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-dbus\") pod \"global-pull-secret-syncer-n8fkx\" (UID: \"1a0016b5-3d41-4c0c-afd6-bf60b5e69237\") " pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:18.494393 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494238    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-multus-socket-dir-parent\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.494393 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494255    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-node-log\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.494393 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494300    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/688032c8-c1d8-4121-8ee0-c224615cc527-cnibin\") pod \"multus-additional-cni-plugins-z26mm\" (UID: \"688032c8-c1d8-4121-8ee0-c224615cc527\") " pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.494393 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494299    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-run-ovn\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.494393 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494336    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-run\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.494393 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494358    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-run-systemd\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.494393 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494400    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-host-slash\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.495135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494426    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-kubelet-config\") pod \"global-pull-secret-syncer-n8fkx\" (UID: \"1a0016b5-3d41-4c0c-afd6-bf60b5e69237\") " pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:18.495135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494457    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkvh6\" (UniqueName: \"kubernetes.io/projected/688032c8-c1d8-4121-8ee0-c224615cc527-kube-api-access-kkvh6\") pod \"multus-additional-cni-plugins-z26mm\" (UID: \"688032c8-c1d8-4121-8ee0-c224615cc527\") " pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.495135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494482    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-run-ovn\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.495135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494484    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e700d4ba-66b4-429a-bae1-74b61fc48253-multus-daemon-config\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.495135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494517    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-var-lib-kubelet\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.495135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494545    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1ad3cc85-481a-4042-bc66-5214457747e0-socket-dir\") pod \"aws-ebs-csi-driver-node-ngvf9\" (UID: \"1ad3cc85-481a-4042-bc66-5214457747e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.495135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494585    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ead9d685-da05-4132-b36c-4887546472f2-tmp-dir\") pod \"node-resolver-nrnz8\" (UID: \"ead9d685-da05-4132-b36c-4887546472f2\") " pod="openshift-dns/node-resolver-nrnz8"
Sep 18 06:22:18.495135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494617    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/688032c8-c1d8-4121-8ee0-c224615cc527-system-cni-dir\") pod \"multus-additional-cni-plugins-z26mm\" (UID: \"688032c8-c1d8-4121-8ee0-c224615cc527\") " pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.495135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494643    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/688032c8-c1d8-4121-8ee0-c224615cc527-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z26mm\" (UID: \"688032c8-c1d8-4121-8ee0-c224615cc527\") " pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.495135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494668    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-host-run-multus-certs\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.495135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494693    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr4xq\" (UniqueName: \"kubernetes.io/projected/ead9d685-da05-4132-b36c-4887546472f2-kube-api-access-rr4xq\") pod \"node-resolver-nrnz8\" (UID: \"ead9d685-da05-4132-b36c-4887546472f2\") " pod="openshift-dns/node-resolver-nrnz8"
Sep 18 06:22:18.495135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494719    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/688032c8-c1d8-4121-8ee0-c224615cc527-cni-binary-copy\") pod \"multus-additional-cni-plugins-z26mm\" (UID: \"688032c8-c1d8-4121-8ee0-c224615cc527\") " pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.495135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494746    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-multus-conf-dir\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.495135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494866    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-etc-kubernetes\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.495135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494905    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d1f8c4d2-31e1-4cca-b763-dc559fa12937-agent-certs\") pod \"konnectivity-agent-2ffp7\" (UID: \"d1f8c4d2-31e1-4cca-b763-dc559fa12937\") " pod="kube-system/konnectivity-agent-2ffp7"
Sep 18 06:22:18.495135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494930    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-hostroot\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.495135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494956    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-host-cni-bin\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.496054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.494998    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-env-overrides\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.496054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495023    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b76b1c21-d297-4f27-bc91-208669dc2890-iptables-alerter-script\") pod \"iptables-alerter-bgkn4\" (UID: \"b76b1c21-d297-4f27-bc91-208669dc2890\") " pod="openshift-network-operator/iptables-alerter-bgkn4"
Sep 18 06:22:18.496054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495051    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-system-cni-dir\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.496054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495075    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-lib-modules\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.496054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495100    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hwvr\" (UniqueName: \"kubernetes.io/projected/612baaf8-db1e-44ca-993e-836959f23b2d-kube-api-access-7hwvr\") pod \"node-ca-6lmr2\" (UID: \"612baaf8-db1e-44ca-993e-836959f23b2d\") " pod="openshift-image-registry/node-ca-6lmr2"
Sep 18 06:22:18.496054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495124    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-host-run-netns\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.496054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495149    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/688032c8-c1d8-4121-8ee0-c224615cc527-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z26mm\" (UID: \"688032c8-c1d8-4121-8ee0-c224615cc527\") " pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.496054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495169    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-host-run-multus-certs\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.496054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495173    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs\") pod \"network-metrics-daemon-k6z9c\" (UID: \"bcf277d6-51cd-4bc8-ba3f-e6fcf272d363\") " pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:18.496054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495500    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e700d4ba-66b4-429a-bae1-74b61fc48253-multus-daemon-config\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.496054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495558    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-etc-kubernetes\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.496054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495594    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-multus-conf-dir\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.496054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495610    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-env-overrides\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.496054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495637    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-host-cni-netd\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.496054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495651    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-etc-modprobe-d\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.496054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495673    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-host-slash\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.496054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495682    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-host\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.496811 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495718    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-kubelet-config\") pod \"global-pull-secret-syncer-n8fkx\" (UID: \"1a0016b5-3d41-4c0c-afd6-bf60b5e69237\") " pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:18.496811 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495712    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1ad3cc85-481a-4042-bc66-5214457747e0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ngvf9\" (UID: \"1ad3cc85-481a-4042-bc66-5214457747e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.496811 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495780    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-system-cni-dir\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.496811 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495780    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/688032c8-c1d8-4121-8ee0-c224615cc527-cni-binary-copy\") pod \"multus-additional-cni-plugins-z26mm\" (UID: \"688032c8-c1d8-4121-8ee0-c224615cc527\") " pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.496811 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495798    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/688032c8-c1d8-4121-8ee0-c224615cc527-system-cni-dir\") pod \"multus-additional-cni-plugins-z26mm\" (UID: \"688032c8-c1d8-4121-8ee0-c224615cc527\") " pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.496811 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495806    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-var-lib-openvswitch\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.496811 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495834    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-host-run-netns\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.496811 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495835    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-etc-openvswitch\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.496811 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495871    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-ovnkube-script-lib\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.496811 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495880    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-etc-openvswitch\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.496811 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495885    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b76b1c21-d297-4f27-bc91-208669dc2890-iptables-alerter-script\") pod \"iptables-alerter-bgkn4\" (UID: \"b76b1c21-d297-4f27-bc91-208669dc2890\") " pod="openshift-network-operator/iptables-alerter-bgkn4"
Sep 18 06:22:18.496811 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495918    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-var-lib-openvswitch\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.496811 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495923    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-multus-cni-dir\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.496811 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495925    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-host-cni-bin\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.496811 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495952    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-etc-sysconfig\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.496811 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495979    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/612baaf8-db1e-44ca-993e-836959f23b2d-host\") pod \"node-ca-6lmr2\" (UID: \"612baaf8-db1e-44ca-993e-836959f23b2d\") " pod="openshift-image-registry/node-ca-6lmr2"
Sep 18 06:22:18.496811 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.495999    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-multus-cni-dir\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.497407 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496039    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-hostroot\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.497407 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496054    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/688032c8-c1d8-4121-8ee0-c224615cc527-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z26mm\" (UID: \"688032c8-c1d8-4121-8ee0-c224615cc527\") " pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.497407 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496097    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-systemd-units\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.497407 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496158    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-host-run-netns\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.497407 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496183    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfk9t\" (UniqueName: \"kubernetes.io/projected/e700d4ba-66b4-429a-bae1-74b61fc48253-kube-api-access-vfk9t\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.497407 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496262    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/688032c8-c1d8-4121-8ee0-c224615cc527-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-z26mm\" (UID: \"688032c8-c1d8-4121-8ee0-c224615cc527\") " pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.497407 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496265    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/688032c8-c1d8-4121-8ee0-c224615cc527-tuning-conf-dir\") pod \"multus-additional-cni-plugins-z26mm\" (UID: \"688032c8-c1d8-4121-8ee0-c224615cc527\") " pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.497407 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496279    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-systemd-units\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.497407 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496297    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-host-run-netns\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.497407 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496442    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-ovnkube-script-lib\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.497407 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496475    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/688032c8-c1d8-4121-8ee0-c224615cc527-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-z26mm\" (UID: \"688032c8-c1d8-4121-8ee0-c224615cc527\") " pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.497407 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496493    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-etc-sysctl-conf\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.497407 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496531    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-etc-systemd\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.497407 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496554    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1ad3cc85-481a-4042-bc66-5214457747e0-registration-dir\") pod \"aws-ebs-csi-driver-node-ngvf9\" (UID: \"1ad3cc85-481a-4042-bc66-5214457747e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.497407 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496580    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.497407 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496603    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-cnibin\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.497407 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496632    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdjn4\" (UniqueName: \"kubernetes.io/projected/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-kube-api-access-tdjn4\") pod \"network-metrics-daemon-k6z9c\" (UID: \"bcf277d6-51cd-4bc8-ba3f-e6fcf272d363\") " pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:18.498070 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496659    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8p5b\" (UniqueName: \"kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b\") pod \"network-check-target-98gq6\" (UID: \"5b9b648c-4c7d-4f02-a6b3-c8e719f972b4\") " pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:18.498070 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496682    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-log-socket\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.498070 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496686    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.498070 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496708    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret\") pod \"global-pull-secret-syncer-n8fkx\" (UID: \"1a0016b5-3d41-4c0c-afd6-bf60b5e69237\") " pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:18.498070 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496732    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b76b1c21-d297-4f27-bc91-208669dc2890-host-slash\") pod \"iptables-alerter-bgkn4\" (UID: \"b76b1c21-d297-4f27-bc91-208669dc2890\") " pod="openshift-network-operator/iptables-alerter-bgkn4"
Sep 18 06:22:18.498070 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496758    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-host-run-ovn-kubernetes\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.498070 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496783    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e700d4ba-66b4-429a-bae1-74b61fc48253-cni-binary-copy\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.498070 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496794    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-log-socket\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.498070 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496807    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-host-run-k8s-cni-cncf-io\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.498070 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496839    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-host-var-lib-cni-bin\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.498070 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496759    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-cnibin\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.498070 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496880    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-host-run-ovn-kubernetes\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.498070 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.496917    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-host-run-k8s-cni-cncf-io\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.498070 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:18.496981    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 06:22:18.498070 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:18.497078    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret podName:1a0016b5-3d41-4c0c-afd6-bf60b5e69237 nodeName:}" failed. No retries permitted until 2026-09-18 06:22:18.997059076 +0000 UTC m=+4.678586798 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret") pod "global-pull-secret-syncer-n8fkx" (UID: "1a0016b5-3d41-4c0c-afd6-bf60b5e69237") : object "kube-system"/"original-pull-secret" not registered
Sep 18 06:22:18.498070 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.497267    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b76b1c21-d297-4f27-bc91-208669dc2890-host-slash\") pod \"iptables-alerter-bgkn4\" (UID: \"b76b1c21-d297-4f27-bc91-208669dc2890\") " pod="openshift-network-operator/iptables-alerter-bgkn4"
Sep 18 06:22:18.498070 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.497286    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e700d4ba-66b4-429a-bae1-74b61fc48253-cni-binary-copy\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.499163 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.497299    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-etc-kubernetes\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.499163 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.497383    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e700d4ba-66b4-429a-bae1-74b61fc48253-host-var-lib-cni-bin\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.499163 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.498432    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-ovn-node-metrics-cert\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.504968 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.504747    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ghww\" (UniqueName: \"kubernetes.io/projected/8cfdc238-d87b-46fb-86d0-e5c13f1019b9-kube-api-access-7ghww\") pod \"ovnkube-node-l6kp8\" (UID: \"8cfdc238-d87b-46fb-86d0-e5c13f1019b9\") " pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.508292 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.505958    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk9h7\" (UniqueName: \"kubernetes.io/projected/b76b1c21-d297-4f27-bc91-208669dc2890-kube-api-access-rk9h7\") pod \"iptables-alerter-bgkn4\" (UID: \"b76b1c21-d297-4f27-bc91-208669dc2890\") " pod="openshift-network-operator/iptables-alerter-bgkn4"
Sep 18 06:22:18.508292 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.507945    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkvh6\" (UniqueName: \"kubernetes.io/projected/688032c8-c1d8-4121-8ee0-c224615cc527-kube-api-access-kkvh6\") pod \"multus-additional-cni-plugins-z26mm\" (UID: \"688032c8-c1d8-4121-8ee0-c224615cc527\") " pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.508877 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.508856    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfk9t\" (UniqueName: \"kubernetes.io/projected/e700d4ba-66b4-429a-bae1-74b61fc48253-kube-api-access-vfk9t\") pod \"multus-nssrf\" (UID: \"e700d4ba-66b4-429a-bae1-74b61fc48253\") " pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.543376 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.543322    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal" event={"ID":"4a8bd5eb6bfe7e7a25b8f8115ad4a547","Type":"ContainerStarted","Data":"8682d3941337917cb49844804c00a76806c8ecc1efb804f04d4f8fe26fd1bf75"}
Sep 18 06:22:18.545179 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.545130    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal" event={"ID":"aab83480badcc6c24454042cf62b9928","Type":"ContainerStarted","Data":"9f82f281f4479143dd582ca77ad5f779f339654a60826c31dd3d20957074e373"}
Sep 18 06:22:18.597897 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.597865    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr4xq\" (UniqueName: \"kubernetes.io/projected/ead9d685-da05-4132-b36c-4887546472f2-kube-api-access-rr4xq\") pod \"node-resolver-nrnz8\" (UID: \"ead9d685-da05-4132-b36c-4887546472f2\") " pod="openshift-dns/node-resolver-nrnz8"
Sep 18 06:22:18.598048 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.597909    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d1f8c4d2-31e1-4cca-b763-dc559fa12937-agent-certs\") pod \"konnectivity-agent-2ffp7\" (UID: \"d1f8c4d2-31e1-4cca-b763-dc559fa12937\") " pod="kube-system/konnectivity-agent-2ffp7"
Sep 18 06:22:18.598048 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.597971    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-lib-modules\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.598048 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.597996    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hwvr\" (UniqueName: \"kubernetes.io/projected/612baaf8-db1e-44ca-993e-836959f23b2d-kube-api-access-7hwvr\") pod \"node-ca-6lmr2\" (UID: \"612baaf8-db1e-44ca-993e-836959f23b2d\") " pod="openshift-image-registry/node-ca-6lmr2"
Sep 18 06:22:18.598235 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598043    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs\") pod \"network-metrics-daemon-k6z9c\" (UID: \"bcf277d6-51cd-4bc8-ba3f-e6fcf272d363\") " pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:18.598235 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:18.598149    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 06:22:18.598235 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598180    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-etc-modprobe-d\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.598235 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:18.598211    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs podName:bcf277d6-51cd-4bc8-ba3f-e6fcf272d363 nodeName:}" failed. No retries permitted until 2026-09-18 06:22:19.09819148 +0000 UTC m=+4.779719214 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs") pod "network-metrics-daemon-k6z9c" (UID: "bcf277d6-51cd-4bc8-ba3f-e6fcf272d363") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 06:22:18.598455 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598253    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-host\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.598455 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598281    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1ad3cc85-481a-4042-bc66-5214457747e0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ngvf9\" (UID: \"1ad3cc85-481a-4042-bc66-5214457747e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.598455 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598285    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-lib-modules\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.598455 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598285    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-etc-modprobe-d\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.598455 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598323    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-host\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.598455 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598346    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1ad3cc85-481a-4042-bc66-5214457747e0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ngvf9\" (UID: \"1ad3cc85-481a-4042-bc66-5214457747e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.598455 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598378    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-etc-sysconfig\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.598455 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598402    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/612baaf8-db1e-44ca-993e-836959f23b2d-host\") pod \"node-ca-6lmr2\" (UID: \"612baaf8-db1e-44ca-993e-836959f23b2d\") " pod="openshift-image-registry/node-ca-6lmr2"
Sep 18 06:22:18.598455 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598430    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-etc-sysctl-conf\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.598455 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598453    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-etc-systemd\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.598918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598467    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-etc-sysconfig\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.598918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598476    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1ad3cc85-481a-4042-bc66-5214457747e0-registration-dir\") pod \"aws-ebs-csi-driver-node-ngvf9\" (UID: \"1ad3cc85-481a-4042-bc66-5214457747e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.598918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598522    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdjn4\" (UniqueName: \"kubernetes.io/projected/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-kube-api-access-tdjn4\") pod \"network-metrics-daemon-k6z9c\" (UID: \"bcf277d6-51cd-4bc8-ba3f-e6fcf272d363\") " pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:18.598918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598524    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/612baaf8-db1e-44ca-993e-836959f23b2d-host\") pod \"node-ca-6lmr2\" (UID: \"612baaf8-db1e-44ca-993e-836959f23b2d\") " pod="openshift-image-registry/node-ca-6lmr2"
Sep 18 06:22:18.598918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598529    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-etc-systemd\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.598918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598551    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8p5b\" (UniqueName: \"kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b\") pod \"network-check-target-98gq6\" (UID: \"5b9b648c-4c7d-4f02-a6b3-c8e719f972b4\") " pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:18.598918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598582    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1ad3cc85-481a-4042-bc66-5214457747e0-registration-dir\") pod \"aws-ebs-csi-driver-node-ngvf9\" (UID: \"1ad3cc85-481a-4042-bc66-5214457747e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.598918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598596    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-etc-kubernetes\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.598918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598625    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1ad3cc85-481a-4042-bc66-5214457747e0-sys-fs\") pod \"aws-ebs-csi-driver-node-ngvf9\" (UID: \"1ad3cc85-481a-4042-bc66-5214457747e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.598918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598650    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqppv\" (UniqueName: \"kubernetes.io/projected/1ad3cc85-481a-4042-bc66-5214457747e0-kube-api-access-nqppv\") pod \"aws-ebs-csi-driver-node-ngvf9\" (UID: \"1ad3cc85-481a-4042-bc66-5214457747e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.598918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598674    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/612baaf8-db1e-44ca-993e-836959f23b2d-serviceca\") pod \"node-ca-6lmr2\" (UID: \"612baaf8-db1e-44ca-993e-836959f23b2d\") " pod="openshift-image-registry/node-ca-6lmr2"
Sep 18 06:22:18.598918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598699    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-etc-sysctl-d\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.598918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598709    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-etc-sysctl-conf\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.598918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598722    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-sys\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.598918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598746    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1ad3cc85-481a-4042-bc66-5214457747e0-device-dir\") pod \"aws-ebs-csi-driver-node-ngvf9\" (UID: \"1ad3cc85-481a-4042-bc66-5214457747e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.598918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598771    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d1f8c4d2-31e1-4cca-b763-dc559fa12937-konnectivity-ca\") pod \"konnectivity-agent-2ffp7\" (UID: \"d1f8c4d2-31e1-4cca-b763-dc559fa12937\") " pod="kube-system/konnectivity-agent-2ffp7"
Sep 18 06:22:18.598918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598780    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1ad3cc85-481a-4042-bc66-5214457747e0-sys-fs\") pod \"aws-ebs-csi-driver-node-ngvf9\" (UID: \"1ad3cc85-481a-4042-bc66-5214457747e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.599734 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598783    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-etc-kubernetes\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.599734 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598793    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ba599bd-531c-4d30-829f-b5cde2f02132-tmp\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.599734 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598839    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26tg7\" (UniqueName: \"kubernetes.io/projected/8ba599bd-531c-4d30-829f-b5cde2f02132-kube-api-access-26tg7\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.599734 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598867    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1ad3cc85-481a-4042-bc66-5214457747e0-etc-selinux\") pod \"aws-ebs-csi-driver-node-ngvf9\" (UID: \"1ad3cc85-481a-4042-bc66-5214457747e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.599734 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598890    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ead9d685-da05-4132-b36c-4887546472f2-hosts-file\") pod \"node-resolver-nrnz8\" (UID: \"ead9d685-da05-4132-b36c-4887546472f2\") " pod="openshift-dns/node-resolver-nrnz8"
Sep 18 06:22:18.599734 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598899    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-etc-sysctl-d\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.599734 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598915    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8ba599bd-531c-4d30-829f-b5cde2f02132-etc-tuned\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.599734 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598951    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-run\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.599734 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.598980    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-var-lib-kubelet\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.599734 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.599010    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1ad3cc85-481a-4042-bc66-5214457747e0-socket-dir\") pod \"aws-ebs-csi-driver-node-ngvf9\" (UID: \"1ad3cc85-481a-4042-bc66-5214457747e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.599734 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.599020    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1ad3cc85-481a-4042-bc66-5214457747e0-device-dir\") pod \"aws-ebs-csi-driver-node-ngvf9\" (UID: \"1ad3cc85-481a-4042-bc66-5214457747e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.599734 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.599032    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ead9d685-da05-4132-b36c-4887546472f2-tmp-dir\") pod \"node-resolver-nrnz8\" (UID: \"ead9d685-da05-4132-b36c-4887546472f2\") " pod="openshift-dns/node-resolver-nrnz8"
Sep 18 06:22:18.599734 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.599121    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-sys\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.599734 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.599325    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ead9d685-da05-4132-b36c-4887546472f2-tmp-dir\") pod \"node-resolver-nrnz8\" (UID: \"ead9d685-da05-4132-b36c-4887546472f2\") " pod="openshift-dns/node-resolver-nrnz8"
Sep 18 06:22:18.599734 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.599410    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/612baaf8-db1e-44ca-993e-836959f23b2d-serviceca\") pod \"node-ca-6lmr2\" (UID: \"612baaf8-db1e-44ca-993e-836959f23b2d\") " pod="openshift-image-registry/node-ca-6lmr2"
Sep 18 06:22:18.599734 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.599426    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-run\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.599734 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.599474    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8ba599bd-531c-4d30-829f-b5cde2f02132-var-lib-kubelet\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.599734 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.599507    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1ad3cc85-481a-4042-bc66-5214457747e0-etc-selinux\") pod \"aws-ebs-csi-driver-node-ngvf9\" (UID: \"1ad3cc85-481a-4042-bc66-5214457747e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.600597 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.599519    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ead9d685-da05-4132-b36c-4887546472f2-hosts-file\") pod \"node-resolver-nrnz8\" (UID: \"ead9d685-da05-4132-b36c-4887546472f2\") " pod="openshift-dns/node-resolver-nrnz8"
Sep 18 06:22:18.600597 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.599523    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1ad3cc85-481a-4042-bc66-5214457747e0-socket-dir\") pod \"aws-ebs-csi-driver-node-ngvf9\" (UID: \"1ad3cc85-481a-4042-bc66-5214457747e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.600597 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.599636    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d1f8c4d2-31e1-4cca-b763-dc559fa12937-konnectivity-ca\") pod \"konnectivity-agent-2ffp7\" (UID: \"d1f8c4d2-31e1-4cca-b763-dc559fa12937\") " pod="kube-system/konnectivity-agent-2ffp7"
Sep 18 06:22:18.601305 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.601285    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ba599bd-531c-4d30-829f-b5cde2f02132-tmp\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.601384 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.601323    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8ba599bd-531c-4d30-829f-b5cde2f02132-etc-tuned\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.601573 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.601554    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d1f8c4d2-31e1-4cca-b763-dc559fa12937-agent-certs\") pod \"konnectivity-agent-2ffp7\" (UID: \"d1f8c4d2-31e1-4cca-b763-dc559fa12937\") " pod="kube-system/konnectivity-agent-2ffp7"
Sep 18 06:22:18.606825 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.606799    2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 06:22:18.613085 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:18.613063    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 06:22:18.613085 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:18.613088    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 06:22:18.613240 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:18.613101    2568 projected.go:196] Error preparing data for projected volume kube-api-access-p8p5b for pod openshift-network-diagnostics/network-check-target-98gq6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 06:22:18.613240 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:18.613165    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b podName:5b9b648c-4c7d-4f02-a6b3-c8e719f972b4 nodeName:}" failed. No retries permitted until 2026-09-18 06:22:19.113147193 +0000 UTC m=+4.794674920 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-p8p5b" (UniqueName: "kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b") pod "network-check-target-98gq6" (UID: "5b9b648c-4c7d-4f02-a6b3-c8e719f972b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 06:22:18.613629 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.613610    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hwvr\" (UniqueName: \"kubernetes.io/projected/612baaf8-db1e-44ca-993e-836959f23b2d-kube-api-access-7hwvr\") pod \"node-ca-6lmr2\" (UID: \"612baaf8-db1e-44ca-993e-836959f23b2d\") " pod="openshift-image-registry/node-ca-6lmr2"
Sep 18 06:22:18.613778 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.613757    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqppv\" (UniqueName: \"kubernetes.io/projected/1ad3cc85-481a-4042-bc66-5214457747e0-kube-api-access-nqppv\") pod \"aws-ebs-csi-driver-node-ngvf9\" (UID: \"1ad3cc85-481a-4042-bc66-5214457747e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.613949 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.613923    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr4xq\" (UniqueName: \"kubernetes.io/projected/ead9d685-da05-4132-b36c-4887546472f2-kube-api-access-rr4xq\") pod \"node-resolver-nrnz8\" (UID: \"ead9d685-da05-4132-b36c-4887546472f2\") " pod="openshift-dns/node-resolver-nrnz8"
Sep 18 06:22:18.614023 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.614011    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdjn4\" (UniqueName: \"kubernetes.io/projected/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-kube-api-access-tdjn4\") pod \"network-metrics-daemon-k6z9c\" (UID: \"bcf277d6-51cd-4bc8-ba3f-e6fcf272d363\") " pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:18.614564 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.614545    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26tg7\" (UniqueName: \"kubernetes.io/projected/8ba599bd-531c-4d30-829f-b5cde2f02132-kube-api-access-26tg7\") pod \"tuned-cqb8p\" (UID: \"8ba599bd-531c-4d30-829f-b5cde2f02132\") " pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.698799 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.698718    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bgkn4"
Sep 18 06:22:18.706920 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.706884    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:18.714866 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.714845    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-z26mm"
Sep 18 06:22:18.722355 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.722337    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nssrf"
Sep 18 06:22:18.728800 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.728784    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cqb8p"
Sep 18 06:22:18.737360 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.737342    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2ffp7"
Sep 18 06:22:18.744901 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.744883    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9"
Sep 18 06:22:18.752493 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.752459    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-nrnz8"
Sep 18 06:22:18.758265 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:18.758121    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6lmr2"
Sep 18 06:22:19.002163 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:19.002074    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret\") pod \"global-pull-secret-syncer-n8fkx\" (UID: \"1a0016b5-3d41-4c0c-afd6-bf60b5e69237\") " pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:19.002306 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:19.002265    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 06:22:19.002364 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:19.002351    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret podName:1a0016b5-3d41-4c0c-afd6-bf60b5e69237 nodeName:}" failed. No retries permitted until 2026-09-18 06:22:20.002329059 +0000 UTC m=+5.683856795 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret") pod "global-pull-secret-syncer-n8fkx" (UID: "1a0016b5-3d41-4c0c-afd6-bf60b5e69237") : object "kube-system"/"original-pull-secret" not registered
Sep 18 06:22:19.102963 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:19.102928    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs\") pod \"network-metrics-daemon-k6z9c\" (UID: \"bcf277d6-51cd-4bc8-ba3f-e6fcf272d363\") " pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:19.103158 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:19.103112    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 06:22:19.103252 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:19.103190    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs podName:bcf277d6-51cd-4bc8-ba3f-e6fcf272d363 nodeName:}" failed. No retries permitted until 2026-09-18 06:22:20.10316984 +0000 UTC m=+5.784697584 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs") pod "network-metrics-daemon-k6z9c" (UID: "bcf277d6-51cd-4bc8-ba3f-e6fcf272d363") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 06:22:19.203600 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:19.203559    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8p5b\" (UniqueName: \"kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b\") pod \"network-check-target-98gq6\" (UID: \"5b9b648c-4c7d-4f02-a6b3-c8e719f972b4\") " pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:19.203770 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:19.203755    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 06:22:19.203827 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:19.203777    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 06:22:19.203827 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:19.203787    2568 projected.go:196] Error preparing data for projected volume kube-api-access-p8p5b for pod openshift-network-diagnostics/network-check-target-98gq6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 06:22:19.203916 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:19.203846    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b podName:5b9b648c-4c7d-4f02-a6b3-c8e719f972b4 nodeName:}" failed. No retries permitted until 2026-09-18 06:22:20.203828934 +0000 UTC m=+5.885356680 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-p8p5b" (UniqueName: "kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b") pod "network-check-target-98gq6" (UID: "5b9b648c-4c7d-4f02-a6b3-c8e719f972b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 06:22:19.426117 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:19.426039    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 06:17:18 +0000 UTC" deadline="2028-03-03 18:39:29.523979827 +0000 UTC"
Sep 18 06:22:19.426117 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:19.426069    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12780h17m10.097914568s"
Sep 18 06:22:19.536774 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:19.536745    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:19.536931 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:19.536750    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:19.536931 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:19.536870    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6z9c" podUID="bcf277d6-51cd-4bc8-ba3f-e6fcf272d363"
Sep 18 06:22:19.537052 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:19.536939    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98gq6" podUID="5b9b648c-4c7d-4f02-a6b3-c8e719f972b4"
Sep 18 06:22:19.654232 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:19.654193    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cfdc238_d87b_46fb_86d0_e5c13f1019b9.slice/crio-623b075d0b658e44224d075a023bdd072ccc777591bb672ebe865dfa76fee7d0 WatchSource:0}: Error finding container 623b075d0b658e44224d075a023bdd072ccc777591bb672ebe865dfa76fee7d0: Status 404 returned error can't find the container with id 623b075d0b658e44224d075a023bdd072ccc777591bb672ebe865dfa76fee7d0
Sep 18 06:22:19.655894 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:19.655562    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podead9d685_da05_4132_b36c_4887546472f2.slice/crio-42465c7d8c4927da8f1323326d5202a2b6a434d7b004e9e4d3a8a33b84cc44b3 WatchSource:0}: Error finding container 42465c7d8c4927da8f1323326d5202a2b6a434d7b004e9e4d3a8a33b84cc44b3: Status 404 returned error can't find the container with id 42465c7d8c4927da8f1323326d5202a2b6a434d7b004e9e4d3a8a33b84cc44b3
Sep 18 06:22:19.659603 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:19.659524    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ad3cc85_481a_4042_bc66_5214457747e0.slice/crio-c0994990b8d145ae3ef405b4b24f86546b5f69976a26e4a32ef98ee6d117d91e WatchSource:0}: Error finding container c0994990b8d145ae3ef405b4b24f86546b5f69976a26e4a32ef98ee6d117d91e: Status 404 returned error can't find the container with id c0994990b8d145ae3ef405b4b24f86546b5f69976a26e4a32ef98ee6d117d91e
Sep 18 06:22:19.661109 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:19.661085    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod612baaf8_db1e_44ca_993e_836959f23b2d.slice/crio-0a4701eaaad0e8fb4256a890aad22d2555fdcf39aaa1ccbd5015bada8538fe30 WatchSource:0}: Error finding container 0a4701eaaad0e8fb4256a890aad22d2555fdcf39aaa1ccbd5015bada8538fe30: Status 404 returned error can't find the container with id 0a4701eaaad0e8fb4256a890aad22d2555fdcf39aaa1ccbd5015bada8538fe30
Sep 18 06:22:19.662091 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:19.662069    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ba599bd_531c_4d30_829f_b5cde2f02132.slice/crio-26e73b0cb71b96b9e2e954cd39cc1019fb5acc0aca521aaf8022859d91578728 WatchSource:0}: Error finding container 26e73b0cb71b96b9e2e954cd39cc1019fb5acc0aca521aaf8022859d91578728: Status 404 returned error can't find the container with id 26e73b0cb71b96b9e2e954cd39cc1019fb5acc0aca521aaf8022859d91578728
Sep 18 06:22:19.663238 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:19.663104    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode700d4ba_66b4_429a_bae1_74b61fc48253.slice/crio-49a39f4fcafab69fc8665f6f40e7ecee809f87a00aee627b69a8e34c785d93c7 WatchSource:0}: Error finding container 49a39f4fcafab69fc8665f6f40e7ecee809f87a00aee627b69a8e34c785d93c7: Status 404 returned error can't find the container with id 49a39f4fcafab69fc8665f6f40e7ecee809f87a00aee627b69a8e34c785d93c7
Sep 18 06:22:19.665750 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:19.665085    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod688032c8_c1d8_4121_8ee0_c224615cc527.slice/crio-689c48678ed60d5e63e2e90fe1cebfdb6e5f125dea00f523620727a0f101f976 WatchSource:0}: Error finding container 689c48678ed60d5e63e2e90fe1cebfdb6e5f125dea00f523620727a0f101f976: Status 404 returned error can't find the container with id 689c48678ed60d5e63e2e90fe1cebfdb6e5f125dea00f523620727a0f101f976
Sep 18 06:22:19.665750 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:19.665290    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb76b1c21_d297_4f27_bc91_208669dc2890.slice/crio-24d0d2f0a876601a6e72a7e1f0b2314bd2c43569a7add1073c86f4efd7db5986 WatchSource:0}: Error finding container 24d0d2f0a876601a6e72a7e1f0b2314bd2c43569a7add1073c86f4efd7db5986: Status 404 returned error can't find the container with id 24d0d2f0a876601a6e72a7e1f0b2314bd2c43569a7add1073c86f4efd7db5986
Sep 18 06:22:19.667792 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:19.667752    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1f8c4d2_31e1_4cca_b763_dc559fa12937.slice/crio-18499bd1b3fc728a41952af03c2efca210c46f7ee051312db94f47901c5c771c WatchSource:0}: Error finding container 18499bd1b3fc728a41952af03c2efca210c46f7ee051312db94f47901c5c771c: Status 404 returned error can't find the container with id 18499bd1b3fc728a41952af03c2efca210c46f7ee051312db94f47901c5c771c
Sep 18 06:22:20.008781 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:20.008522    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret\") pod \"global-pull-secret-syncer-n8fkx\" (UID: \"1a0016b5-3d41-4c0c-afd6-bf60b5e69237\") " pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:20.008781 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:20.008649    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 06:22:20.008941 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:20.008794    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret podName:1a0016b5-3d41-4c0c-afd6-bf60b5e69237 nodeName:}" failed. No retries permitted until 2026-09-18 06:22:22.008780016 +0000 UTC m=+7.690307751 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret") pod "global-pull-secret-syncer-n8fkx" (UID: "1a0016b5-3d41-4c0c-afd6-bf60b5e69237") : object "kube-system"/"original-pull-secret" not registered
Sep 18 06:22:20.109893 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:20.109821    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs\") pod \"network-metrics-daemon-k6z9c\" (UID: \"bcf277d6-51cd-4bc8-ba3f-e6fcf272d363\") " pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:20.110075 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:20.109977    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 06:22:20.110075 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:20.110044    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs podName:bcf277d6-51cd-4bc8-ba3f-e6fcf272d363 nodeName:}" failed. No retries permitted until 2026-09-18 06:22:22.110025169 +0000 UTC m=+7.791552908 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs") pod "network-metrics-daemon-k6z9c" (UID: "bcf277d6-51cd-4bc8-ba3f-e6fcf272d363") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 06:22:20.210543 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:20.210485    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8p5b\" (UniqueName: \"kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b\") pod \"network-check-target-98gq6\" (UID: \"5b9b648c-4c7d-4f02-a6b3-c8e719f972b4\") " pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:20.210717 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:20.210668    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 06:22:20.210717 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:20.210690    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 06:22:20.210717 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:20.210702    2568 projected.go:196] Error preparing data for projected volume kube-api-access-p8p5b for pod openshift-network-diagnostics/network-check-target-98gq6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 06:22:20.210878 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:20.210751    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b podName:5b9b648c-4c7d-4f02-a6b3-c8e719f972b4 nodeName:}" failed. No retries permitted until 2026-09-18 06:22:22.210738689 +0000 UTC m=+7.892266424 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-p8p5b" (UniqueName: "kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b") pod "network-check-target-98gq6" (UID: "5b9b648c-4c7d-4f02-a6b3-c8e719f972b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 06:22:20.427011 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:20.426899    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 06:17:18 +0000 UTC" deadline="2028-04-02 21:43:49.840324854 +0000 UTC"
Sep 18 06:22:20.427011 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:20.426964    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13503h21m29.41336486s"
Sep 18 06:22:20.538456 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:20.538427    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:20.538616 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:20.538560    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8fkx" podUID="1a0016b5-3d41-4c0c-afd6-bf60b5e69237"
Sep 18 06:22:20.568031 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:20.567986    2568 generic.go:356] "Generic (PLEG): container finished" podID="4a8bd5eb6bfe7e7a25b8f8115ad4a547" containerID="1d631daf7070b6a57883b5c93109c4d3e861c61babeca3e04af0a50cab53807e" exitCode=0
Sep 18 06:22:20.568183 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:20.568072    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal" event={"ID":"4a8bd5eb6bfe7e7a25b8f8115ad4a547","Type":"ContainerDied","Data":"1d631daf7070b6a57883b5c93109c4d3e861c61babeca3e04af0a50cab53807e"}
Sep 18 06:22:20.577497 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:20.577463    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6lmr2" event={"ID":"612baaf8-db1e-44ca-993e-836959f23b2d","Type":"ContainerStarted","Data":"0a4701eaaad0e8fb4256a890aad22d2555fdcf39aaa1ccbd5015bada8538fe30"}
Sep 18 06:22:20.585007 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:20.584920    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8" event={"ID":"8cfdc238-d87b-46fb-86d0-e5c13f1019b9","Type":"ContainerStarted","Data":"623b075d0b658e44224d075a023bdd072ccc777591bb672ebe865dfa76fee7d0"}
Sep 18 06:22:20.597738 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:20.597687    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal" event={"ID":"aab83480badcc6c24454042cf62b9928","Type":"ContainerStarted","Data":"60ff3212a304f683044480d5479810159b108326ed8f6e68b3051731c7c3448b"}
Sep 18 06:22:20.615481 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:20.615454    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2ffp7" event={"ID":"d1f8c4d2-31e1-4cca-b763-dc559fa12937","Type":"ContainerStarted","Data":"18499bd1b3fc728a41952af03c2efca210c46f7ee051312db94f47901c5c771c"}
Sep 18 06:22:20.626704 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:20.626655    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bgkn4" event={"ID":"b76b1c21-d297-4f27-bc91-208669dc2890","Type":"ContainerStarted","Data":"24d0d2f0a876601a6e72a7e1f0b2314bd2c43569a7add1073c86f4efd7db5986"}
Sep 18 06:22:20.640291 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:20.640267    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z26mm" event={"ID":"688032c8-c1d8-4121-8ee0-c224615cc527","Type":"ContainerStarted","Data":"689c48678ed60d5e63e2e90fe1cebfdb6e5f125dea00f523620727a0f101f976"}
Sep 18 06:22:20.647261 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:20.647173    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nssrf" event={"ID":"e700d4ba-66b4-429a-bae1-74b61fc48253","Type":"ContainerStarted","Data":"49a39f4fcafab69fc8665f6f40e7ecee809f87a00aee627b69a8e34c785d93c7"}
Sep 18 06:22:20.652139 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:20.652094    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cqb8p" event={"ID":"8ba599bd-531c-4d30-829f-b5cde2f02132","Type":"ContainerStarted","Data":"26e73b0cb71b96b9e2e954cd39cc1019fb5acc0aca521aaf8022859d91578728"}
Sep 18 06:22:20.680621 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:20.680553    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9" event={"ID":"1ad3cc85-481a-4042-bc66-5214457747e0","Type":"ContainerStarted","Data":"c0994990b8d145ae3ef405b4b24f86546b5f69976a26e4a32ef98ee6d117d91e"}
Sep 18 06:22:20.694838 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:20.694792    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nrnz8" event={"ID":"ead9d685-da05-4132-b36c-4887546472f2","Type":"ContainerStarted","Data":"42465c7d8c4927da8f1323326d5202a2b6a434d7b004e9e4d3a8a33b84cc44b3"}
Sep 18 06:22:21.536660 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:21.536624    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:21.537133 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:21.536755    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98gq6" podUID="5b9b648c-4c7d-4f02-a6b3-c8e719f972b4"
Sep 18 06:22:21.537193 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:21.537146    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:21.537417 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:21.537354    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6z9c" podUID="bcf277d6-51cd-4bc8-ba3f-e6fcf272d363"
Sep 18 06:22:21.747291 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:21.746312    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal" event={"ID":"4a8bd5eb6bfe7e7a25b8f8115ad4a547","Type":"ContainerStarted","Data":"88850fe65cf95857f407fc52291525e17c717b314d72e9eac7755680a9e03344"}
Sep 18 06:22:21.764323 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:21.764267    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal" podStartSLOduration=4.764248301 podStartE2EDuration="4.764248301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 06:22:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:22:20.620786639 +0000 UTC m=+6.302314384" watchObservedRunningTime="2026-09-18 06:22:21.764248301 +0000 UTC m=+7.445776047"
Sep 18 06:22:21.764496 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:21.764358    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal" podStartSLOduration=4.764351593 podStartE2EDuration="4.764351593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 06:22:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:22:21.76324541 +0000 UTC m=+7.444773157" watchObservedRunningTime="2026-09-18 06:22:21.764351593 +0000 UTC m=+7.445879319"
Sep 18 06:22:22.026408 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:22.025780    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret\") pod \"global-pull-secret-syncer-n8fkx\" (UID: \"1a0016b5-3d41-4c0c-afd6-bf60b5e69237\") " pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:22.026408 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:22.025946    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 06:22:22.026408 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:22.026007    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret podName:1a0016b5-3d41-4c0c-afd6-bf60b5e69237 nodeName:}" failed. No retries permitted until 2026-09-18 06:22:26.025989424 +0000 UTC m=+11.707517151 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret") pod "global-pull-secret-syncer-n8fkx" (UID: "1a0016b5-3d41-4c0c-afd6-bf60b5e69237") : object "kube-system"/"original-pull-secret" not registered
Sep 18 06:22:22.128760 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:22.127055    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs\") pod \"network-metrics-daemon-k6z9c\" (UID: \"bcf277d6-51cd-4bc8-ba3f-e6fcf272d363\") " pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:22.128760 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:22.127200    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 06:22:22.128760 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:22.127283    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs podName:bcf277d6-51cd-4bc8-ba3f-e6fcf272d363 nodeName:}" failed. No retries permitted until 2026-09-18 06:22:26.127263264 +0000 UTC m=+11.808790999 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs") pod "network-metrics-daemon-k6z9c" (UID: "bcf277d6-51cd-4bc8-ba3f-e6fcf272d363") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 06:22:22.228149 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:22.227547    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8p5b\" (UniqueName: \"kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b\") pod \"network-check-target-98gq6\" (UID: \"5b9b648c-4c7d-4f02-a6b3-c8e719f972b4\") " pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:22.228149 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:22.227718    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 06:22:22.228149 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:22.227741    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 06:22:22.228149 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:22.227755    2568 projected.go:196] Error preparing data for projected volume kube-api-access-p8p5b for pod openshift-network-diagnostics/network-check-target-98gq6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 06:22:22.228149 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:22.227811    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b podName:5b9b648c-4c7d-4f02-a6b3-c8e719f972b4 nodeName:}" failed. No retries permitted until 2026-09-18 06:22:26.227794518 +0000 UTC m=+11.909322254 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-p8p5b" (UniqueName: "kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b") pod "network-check-target-98gq6" (UID: "5b9b648c-4c7d-4f02-a6b3-c8e719f972b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 06:22:22.539951 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:22.539429    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:22.539951 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:22.539593    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8fkx" podUID="1a0016b5-3d41-4c0c-afd6-bf60b5e69237"
Sep 18 06:22:23.536754 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:23.536720    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:23.536942 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:23.536848    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98gq6" podUID="5b9b648c-4c7d-4f02-a6b3-c8e719f972b4"
Sep 18 06:22:23.536942 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:23.536885    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:23.537066 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:23.536984    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6z9c" podUID="bcf277d6-51cd-4bc8-ba3f-e6fcf272d363"
Sep 18 06:22:24.536665 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:24.536629    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:24.537136 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:24.536769    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8fkx" podUID="1a0016b5-3d41-4c0c-afd6-bf60b5e69237"
Sep 18 06:22:25.537322 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:25.537283    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:25.537322 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:25.537321    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:25.537807 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:25.537418    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98gq6" podUID="5b9b648c-4c7d-4f02-a6b3-c8e719f972b4"
Sep 18 06:22:25.537893 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:25.537863    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6z9c" podUID="bcf277d6-51cd-4bc8-ba3f-e6fcf272d363"
Sep 18 06:22:26.055200 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:26.055159    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret\") pod \"global-pull-secret-syncer-n8fkx\" (UID: \"1a0016b5-3d41-4c0c-afd6-bf60b5e69237\") " pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:26.055377 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:26.055341    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 06:22:26.055436 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:26.055422    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret podName:1a0016b5-3d41-4c0c-afd6-bf60b5e69237 nodeName:}" failed. No retries permitted until 2026-09-18 06:22:34.055400044 +0000 UTC m=+19.736927778 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret") pod "global-pull-secret-syncer-n8fkx" (UID: "1a0016b5-3d41-4c0c-afd6-bf60b5e69237") : object "kube-system"/"original-pull-secret" not registered
Sep 18 06:22:26.155711 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:26.155665    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs\") pod \"network-metrics-daemon-k6z9c\" (UID: \"bcf277d6-51cd-4bc8-ba3f-e6fcf272d363\") " pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:26.155893 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:26.155872    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 06:22:26.155971 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:26.155949    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs podName:bcf277d6-51cd-4bc8-ba3f-e6fcf272d363 nodeName:}" failed. No retries permitted until 2026-09-18 06:22:34.155929283 +0000 UTC m=+19.837457017 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs") pod "network-metrics-daemon-k6z9c" (UID: "bcf277d6-51cd-4bc8-ba3f-e6fcf272d363") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 06:22:26.257078 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:26.257038    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8p5b\" (UniqueName: \"kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b\") pod \"network-check-target-98gq6\" (UID: \"5b9b648c-4c7d-4f02-a6b3-c8e719f972b4\") " pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:26.257285 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:26.257257    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 06:22:26.257285 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:26.257281    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 06:22:26.257381 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:26.257291    2568 projected.go:196] Error preparing data for projected volume kube-api-access-p8p5b for pod openshift-network-diagnostics/network-check-target-98gq6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 06:22:26.257381 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:26.257350    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b podName:5b9b648c-4c7d-4f02-a6b3-c8e719f972b4 nodeName:}" failed. No retries permitted until 2026-09-18 06:22:34.257333566 +0000 UTC m=+19.938861303 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-p8p5b" (UniqueName: "kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b") pod "network-check-target-98gq6" (UID: "5b9b648c-4c7d-4f02-a6b3-c8e719f972b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 06:22:26.537425 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:26.537355    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:26.537801 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:26.537471    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8fkx" podUID="1a0016b5-3d41-4c0c-afd6-bf60b5e69237"
Sep 18 06:22:27.537449 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:27.537366    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:27.537899 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:27.537380    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:27.537899 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:27.537513    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98gq6" podUID="5b9b648c-4c7d-4f02-a6b3-c8e719f972b4"
Sep 18 06:22:27.537899 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:27.537584    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6z9c" podUID="bcf277d6-51cd-4bc8-ba3f-e6fcf272d363"
Sep 18 06:22:28.536878 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:28.536788    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:28.537153 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:28.536930    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8fkx" podUID="1a0016b5-3d41-4c0c-afd6-bf60b5e69237"
Sep 18 06:22:29.537237 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:29.537072    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:29.537237 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:29.537193    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98gq6" podUID="5b9b648c-4c7d-4f02-a6b3-c8e719f972b4"
Sep 18 06:22:29.537640 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:29.537588    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:29.538318 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:29.537689    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6z9c" podUID="bcf277d6-51cd-4bc8-ba3f-e6fcf272d363"
Sep 18 06:22:29.767986 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:29.766902    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2ffp7" event={"ID":"d1f8c4d2-31e1-4cca-b763-dc559fa12937","Type":"ContainerStarted","Data":"35a9e980d79ef53e46ee6e778039d2b731351bf626242db1d9ec0281a2aebf00"}
Sep 18 06:22:29.768682 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:29.768628    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z26mm" event={"ID":"688032c8-c1d8-4121-8ee0-c224615cc527","Type":"ContainerStarted","Data":"546a3bf86d8425df3ace2604c51da804167db39a32bba107839aac4a0d604e70"}
Sep 18 06:22:29.770661 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:29.770628    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cqb8p" event={"ID":"8ba599bd-531c-4d30-829f-b5cde2f02132","Type":"ContainerStarted","Data":"049c5a60d57b3c4e85fa38201c7fe50e1b4b560e7837a421e0a81f72479d01b6"}
Sep 18 06:22:29.772912 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:29.772862    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9" event={"ID":"1ad3cc85-481a-4042-bc66-5214457747e0","Type":"ContainerStarted","Data":"a90afc0ad18532d3280d79c9dc501d54be05243e37e57eb4ad58ec6525d9563d"}
Sep 18 06:22:29.774996 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:29.774970    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-nrnz8" event={"ID":"ead9d685-da05-4132-b36c-4887546472f2","Type":"ContainerStarted","Data":"18dbe8ffc0b9c67b1d7a87df1c47f5b24f73cb16a63d76e1da589aed12dec487"}
Sep 18 06:22:29.777524 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:29.777504    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6lmr2" event={"ID":"612baaf8-db1e-44ca-993e-836959f23b2d","Type":"ContainerStarted","Data":"6c5877e02beaca4ab10421442773d872e85fcc72c5842f6abae72b076c8dfeb4"}
Sep 18 06:22:29.798954 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:29.798906    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2ffp7" podStartSLOduration=4.001713957 podStartE2EDuration="13.798887721s" totalImagesPullingTime="9.797173764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:22:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:22:29.780481143 +0000 UTC m=+15.462008892" watchObservedRunningTime="2026-09-18 06:22:29.798887721 +0000 UTC m=+15.480415466"
Sep 18 06:22:29.829184 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:29.829134    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cqb8p" podStartSLOduration=3.955237939 podStartE2EDuration="13.829116509s" totalImagesPullingTime="9.87387857s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 06:22:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:22:29.828962712 +0000 UTC m=+15.510490457" watchObservedRunningTime="2026-09-18 06:22:29.829116509 +0000 UTC m=+15.510644254"
Sep 18 06:22:29.829328 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:29.829303    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-nrnz8" podStartSLOduration=4.015495218 podStartE2EDuration="13.829294312s" totalImagesPullingTime="9.813799094s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 06:22:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:22:29.813807651 +0000 UTC m=+15.495335395" watchObservedRunningTime="2026-09-18 06:22:29.829294312 +0000 UTC m=+15.510822057"
Sep 18 06:22:29.841743 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:29.841699    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6lmr2" podStartSLOduration=4.033916811 podStartE2EDuration="13.841690333s" totalImagesPullingTime="9.807773522s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 06:22:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:22:29.841574754 +0000 UTC m=+15.523102509" watchObservedRunningTime="2026-09-18 06:22:29.841690333 +0000 UTC m=+15.523218076"
Sep 18 06:22:30.039954 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:30.039869    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2ffp7"
Sep 18 06:22:30.040588 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:30.040566    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2ffp7"
Sep 18 06:22:30.537675 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:30.537640    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:30.538103 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:30.537780    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8fkx" podUID="1a0016b5-3d41-4c0c-afd6-bf60b5e69237"
Sep 18 06:22:30.781516 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:30.781424    2568 generic.go:356] "Generic (PLEG): container finished" podID="688032c8-c1d8-4121-8ee0-c224615cc527" containerID="546a3bf86d8425df3ace2604c51da804167db39a32bba107839aac4a0d604e70" exitCode=0
Sep 18 06:22:30.781672 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:30.781519    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z26mm" event={"ID":"688032c8-c1d8-4121-8ee0-c224615cc527","Type":"ContainerDied","Data":"546a3bf86d8425df3ace2604c51da804167db39a32bba107839aac4a0d604e70"}
Sep 18 06:22:30.782340 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:30.782312    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2ffp7"
Sep 18 06:22:30.783070 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:30.782977    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2ffp7"
Sep 18 06:22:31.537023 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:31.536945    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:31.537023 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:31.536979    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:31.537233 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:31.537092    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6z9c" podUID="bcf277d6-51cd-4bc8-ba3f-e6fcf272d363"
Sep 18 06:22:31.537291 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:31.537250    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98gq6" podUID="5b9b648c-4c7d-4f02-a6b3-c8e719f972b4"
Sep 18 06:22:31.785804 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:31.785766    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bgkn4" event={"ID":"b76b1c21-d297-4f27-bc91-208669dc2890","Type":"ContainerStarted","Data":"b5e32d532551f934bd85dd88f13013bf972b3672f6967270f55c596e8328498f"}
Sep 18 06:22:31.799427 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:31.799330    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bgkn4" podStartSLOduration=5.968371746 podStartE2EDuration="15.799316019s" totalImagesPullingTime="9.830944273s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 06:22:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:22:31.799131366 +0000 UTC m=+17.480659111" watchObservedRunningTime="2026-09-18 06:22:31.799316019 +0000 UTC m=+17.480843763"
Sep 18 06:22:32.537540 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:32.537319    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:32.537716 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:32.537633    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8fkx" podUID="1a0016b5-3d41-4c0c-afd6-bf60b5e69237"
Sep 18 06:22:33.536729 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:33.536697    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:33.537184 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:33.536808    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98gq6" podUID="5b9b648c-4c7d-4f02-a6b3-c8e719f972b4"
Sep 18 06:22:33.537184 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:33.536841    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:33.537184 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:33.536927    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6z9c" podUID="bcf277d6-51cd-4bc8-ba3f-e6fcf272d363"
Sep 18 06:22:34.118087 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:34.118045    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret\") pod \"global-pull-secret-syncer-n8fkx\" (UID: \"1a0016b5-3d41-4c0c-afd6-bf60b5e69237\") " pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:34.118336 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:34.118179    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 06:22:34.118336 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:34.118256    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret podName:1a0016b5-3d41-4c0c-afd6-bf60b5e69237 nodeName:}" failed. No retries permitted until 2026-09-18 06:22:50.118238489 +0000 UTC m=+35.799766229 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret") pod "global-pull-secret-syncer-n8fkx" (UID: "1a0016b5-3d41-4c0c-afd6-bf60b5e69237") : object "kube-system"/"original-pull-secret" not registered
Sep 18 06:22:34.218429 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:34.218388    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs\") pod \"network-metrics-daemon-k6z9c\" (UID: \"bcf277d6-51cd-4bc8-ba3f-e6fcf272d363\") " pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:34.218603 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:34.218506    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 06:22:34.218603 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:34.218575    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs podName:bcf277d6-51cd-4bc8-ba3f-e6fcf272d363 nodeName:}" failed. No retries permitted until 2026-09-18 06:22:50.218555935 +0000 UTC m=+35.900083671 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs") pod "network-metrics-daemon-k6z9c" (UID: "bcf277d6-51cd-4bc8-ba3f-e6fcf272d363") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 06:22:34.319697 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:34.319657    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8p5b\" (UniqueName: \"kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b\") pod \"network-check-target-98gq6\" (UID: \"5b9b648c-4c7d-4f02-a6b3-c8e719f972b4\") " pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:34.319875 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:34.319821    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 06:22:34.319875 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:34.319840    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 06:22:34.319875 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:34.319851    2568 projected.go:196] Error preparing data for projected volume kube-api-access-p8p5b for pod openshift-network-diagnostics/network-check-target-98gq6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 06:22:34.320004 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:34.319918    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b podName:5b9b648c-4c7d-4f02-a6b3-c8e719f972b4 nodeName:}" failed. No retries permitted until 2026-09-18 06:22:50.319904581 +0000 UTC m=+36.001432316 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-p8p5b" (UniqueName: "kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b") pod "network-check-target-98gq6" (UID: "5b9b648c-4c7d-4f02-a6b3-c8e719f972b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 06:22:34.536959 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:34.536876    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:34.537518 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:34.537012    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8fkx" podUID="1a0016b5-3d41-4c0c-afd6-bf60b5e69237"
Sep 18 06:22:35.412758 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:35.412722    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:35.507947 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:35.507909    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:35.536856 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:35.536822    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:35.537022 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:35.536833    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:35.537022 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:35.536928    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98gq6" podUID="5b9b648c-4c7d-4f02-a6b3-c8e719f972b4"
Sep 18 06:22:35.537022 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:35.537010    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6z9c" podUID="bcf277d6-51cd-4bc8-ba3f-e6fcf272d363"
Sep 18 06:22:36.537714 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:36.537680    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:36.538104 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:36.537804    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8fkx" podUID="1a0016b5-3d41-4c0c-afd6-bf60b5e69237"
Sep 18 06:22:37.536906 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:37.536870    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:37.537043 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:37.536870    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:37.537043 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:37.536986    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6z9c" podUID="bcf277d6-51cd-4bc8-ba3f-e6fcf272d363"
Sep 18 06:22:37.537043 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:37.537020    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98gq6" podUID="5b9b648c-4c7d-4f02-a6b3-c8e719f972b4"
Sep 18 06:22:37.802730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:37.802521    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8" event={"ID":"8cfdc238-d87b-46fb-86d0-e5c13f1019b9","Type":"ContainerStarted","Data":"c96b6a69c9725c9aa0f34c3085567a37eec983b22ea119600f6b356f3104e5a5"}
Sep 18 06:22:37.807793 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:37.807761    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nssrf" event={"ID":"e700d4ba-66b4-429a-bae1-74b61fc48253","Type":"ContainerStarted","Data":"4fa0de435e2d9b946307df7d280372eaf4ec196d7c7c9a8008038d7f043b703b"}
Sep 18 06:22:37.897364 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:37.897334    2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 18 06:22:37.941905 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:37.941876    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:38.044096 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:38.044049    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:38.360143 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:38.360061    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:38.411632 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:38.411587    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:38.471131 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:38.470987    2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-18T06:22:37.897347729Z","UUID":"abb1b7b1-897a-45be-9ffb-64816bbfe3db","Handler":null,"Name":"","Endpoint":""}
Sep 18 06:22:38.472750 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:38.472727    2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 18 06:22:38.472898 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:38.472763    2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 18 06:22:38.537348 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:38.537305    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:38.537507 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:38.537428    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8fkx" podUID="1a0016b5-3d41-4c0c-afd6-bf60b5e69237"
Sep 18 06:22:38.814299 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:38.814050    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9" event={"ID":"1ad3cc85-481a-4042-bc66-5214457747e0","Type":"ContainerStarted","Data":"fe631e69584bbf7576bdd374f081d9c57ddf2b0f25c6db52db3d00938d0ade91"}
Sep 18 06:22:38.816895 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:38.816871    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8" event={"ID":"8cfdc238-d87b-46fb-86d0-e5c13f1019b9","Type":"ContainerStarted","Data":"7630f5ee581d10af7fcc789bc135224f7feb99ea12b4613056c3474d72034ccc"}
Sep 18 06:22:38.817012 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:38.816931    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8" event={"ID":"8cfdc238-d87b-46fb-86d0-e5c13f1019b9","Type":"ContainerStarted","Data":"205ef71561e06fa42bbcc6f7069c26757b0b60c75d5dfbb160985b8073c82202"}
Sep 18 06:22:38.817012 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:38.816962    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8" event={"ID":"8cfdc238-d87b-46fb-86d0-e5c13f1019b9","Type":"ContainerStarted","Data":"d2c8ec5ba658ce907897996a95ee2bde172199876d694f9241f3b64127e27095"}
Sep 18 06:22:38.817012 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:38.816990    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8" event={"ID":"8cfdc238-d87b-46fb-86d0-e5c13f1019b9","Type":"ContainerStarted","Data":"273794faf52966a8379e0a3abbf4e883d4482c9faf165144a6d2fdf9c079b826"}
Sep 18 06:22:38.817111 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:38.817021    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8" event={"ID":"8cfdc238-d87b-46fb-86d0-e5c13f1019b9","Type":"ContainerStarted","Data":"18f528866de23189607a7e04025036195db386896121c6ca1c81be232be0fa10"}
Sep 18 06:22:39.537007 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:39.536971    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:39.537007 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:39.537015    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:39.537282 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:39.537106    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6z9c" podUID="bcf277d6-51cd-4bc8-ba3f-e6fcf272d363"
Sep 18 06:22:39.537282 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:39.537261    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98gq6" podUID="5b9b648c-4c7d-4f02-a6b3-c8e719f972b4"
Sep 18 06:22:39.821732 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:39.821649    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9" event={"ID":"1ad3cc85-481a-4042-bc66-5214457747e0","Type":"ContainerStarted","Data":"f8a136cecbb013fddf37e01e38eddba5468d24b186f1f8adf589dc49c5cf0b83"}
Sep 18 06:22:39.838555 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:39.838510    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ngvf9" podStartSLOduration=5.02771947 podStartE2EDuration="23.83849686s" totalImagesPullingTime="18.81077739s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 06:22:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:22:39.838373402 +0000 UTC m=+25.519901146" watchObservedRunningTime="2026-09-18 06:22:39.83849686 +0000 UTC m=+25.520024604"
Sep 18 06:22:39.839042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:39.839020    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nssrf" podStartSLOduration=5.868903749 podStartE2EDuration="23.839010818s" totalImagesPullingTime="17.970107069s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 06:22:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:22:37.823432438 +0000 UTC m=+23.504960183" watchObservedRunningTime="2026-09-18 06:22:39.839010818 +0000 UTC m=+25.520538562"
Sep 18 06:22:40.282272 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:40.282209    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:40.330205 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:40.330152    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:40.540766 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:40.540699    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:40.540922 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:40.540802    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8fkx" podUID="1a0016b5-3d41-4c0c-afd6-bf60b5e69237"
Sep 18 06:22:40.827829 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:40.827752    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8" event={"ID":"8cfdc238-d87b-46fb-86d0-e5c13f1019b9","Type":"ContainerStarted","Data":"900d611275dd9f0825ed172c760990d89227ea707bc1fd8334d841a438abe3f2"}
Sep 18 06:22:41.537047 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:41.537016    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:41.537047 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:41.537056    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:41.537298 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:41.537140    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6z9c" podUID="bcf277d6-51cd-4bc8-ba3f-e6fcf272d363"
Sep 18 06:22:41.537340 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:41.537300    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98gq6" podUID="5b9b648c-4c7d-4f02-a6b3-c8e719f972b4"
Sep 18 06:22:41.621455 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:41.621424    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:41.694149 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:41.694106    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:41.789894 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:41.789810    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:42.537526 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:42.537492    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:42.538041 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:42.537598    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8fkx" podUID="1a0016b5-3d41-4c0c-afd6-bf60b5e69237"
Sep 18 06:22:42.683162 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:42.683079    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:42.731173 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:42.731141    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:42.835255 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:42.835208    2568 generic.go:356] "Generic (PLEG): container finished" podID="688032c8-c1d8-4121-8ee0-c224615cc527" containerID="90519b8382297a07a9ad307cd1ebc72e182e838ebe5bbb719134869dbdf61eb9" exitCode=0
Sep 18 06:22:42.835255 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:42.835248    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z26mm" event={"ID":"688032c8-c1d8-4121-8ee0-c224615cc527","Type":"ContainerDied","Data":"90519b8382297a07a9ad307cd1ebc72e182e838ebe5bbb719134869dbdf61eb9"}
Sep 18 06:22:42.839243 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:42.839201    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8" event={"ID":"8cfdc238-d87b-46fb-86d0-e5c13f1019b9","Type":"ContainerStarted","Data":"7f37cbf4ae45d596717f6b572babad810b0f25ec5bbb5785c81533035f0edde4"}
Sep 18 06:22:42.839505 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:42.839484    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:42.839639 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:42.839569    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:42.853894 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:42.853873    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:42.876201 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:42.876157    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8" podStartSLOduration=8.949186702 podStartE2EDuration="26.876142364s" totalImagesPullingTime="17.926955662s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 06:22:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:22:42.87460392 +0000 UTC m=+28.556131666" watchObservedRunningTime="2026-09-18 06:22:42.876142364 +0000 UTC m=+28.557670107"
Sep 18 06:22:43.536536 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:43.536500    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:43.536702 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:43.536551    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:43.536702 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:43.536627    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98gq6" podUID="5b9b648c-4c7d-4f02-a6b3-c8e719f972b4"
Sep 18 06:22:43.536809 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:43.536753    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6z9c" podUID="bcf277d6-51cd-4bc8-ba3f-e6fcf272d363"
Sep 18 06:22:43.842492 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:43.842416    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:43.858647 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:43.858621    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:22:44.175482 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:44.175247    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-98gq6"]
Sep 18 06:22:44.175824 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:44.175793    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:44.176038 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:44.175898    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98gq6" podUID="5b9b648c-4c7d-4f02-a6b3-c8e719f972b4"
Sep 18 06:22:44.176715 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:44.176681    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k6z9c"]
Sep 18 06:22:44.176843 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:44.176796    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:44.177047 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:44.176890    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6z9c" podUID="bcf277d6-51cd-4bc8-ba3f-e6fcf272d363"
Sep 18 06:22:44.177378 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:44.177358    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n8fkx"]
Sep 18 06:22:44.177481 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:44.177432    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:44.177540 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:44.177510    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8fkx" podUID="1a0016b5-3d41-4c0c-afd6-bf60b5e69237"
Sep 18 06:22:44.846036 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:44.846004    2568 generic.go:356] "Generic (PLEG): container finished" podID="688032c8-c1d8-4121-8ee0-c224615cc527" containerID="646d19adbaf34648a777acc7e9696d59db52e86602dc851c753964355bc9b21f" exitCode=0
Sep 18 06:22:44.846428 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:44.846094    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z26mm" event={"ID":"688032c8-c1d8-4121-8ee0-c224615cc527","Type":"ContainerDied","Data":"646d19adbaf34648a777acc7e9696d59db52e86602dc851c753964355bc9b21f"}
Sep 18 06:22:45.536902 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:45.536863    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:45.537049 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:45.536909    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:45.537049 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:45.536966    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:45.537049 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:45.536983    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98gq6" podUID="5b9b648c-4c7d-4f02-a6b3-c8e719f972b4"
Sep 18 06:22:45.537049 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:45.537041    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6z9c" podUID="bcf277d6-51cd-4bc8-ba3f-e6fcf272d363"
Sep 18 06:22:45.537248 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:45.537101    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8fkx" podUID="1a0016b5-3d41-4c0c-afd6-bf60b5e69237"
Sep 18 06:22:46.685865 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:46.685827    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:46.852817 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:46.852789    2568 generic.go:356] "Generic (PLEG): container finished" podID="688032c8-c1d8-4121-8ee0-c224615cc527" containerID="4cdffae51c37278a82bea60eb2cde5c8abf4c18d245902209f871d1dc1e4a0c1" exitCode=0
Sep 18 06:22:46.852972 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:46.852835    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z26mm" event={"ID":"688032c8-c1d8-4121-8ee0-c224615cc527","Type":"ContainerDied","Data":"4cdffae51c37278a82bea60eb2cde5c8abf4c18d245902209f871d1dc1e4a0c1"}
Sep 18 06:22:47.537178 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:47.537151    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:47.537373 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:47.537151    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:47.537373 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:47.537151    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:47.537373 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:47.537296    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6z9c" podUID="bcf277d6-51cd-4bc8-ba3f-e6fcf272d363"
Sep 18 06:22:47.537538 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:47.537375    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8fkx" podUID="1a0016b5-3d41-4c0c-afd6-bf60b5e69237"
Sep 18 06:22:47.537538 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:47.537452    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98gq6" podUID="5b9b648c-4c7d-4f02-a6b3-c8e719f972b4"
Sep 18 06:22:49.536744 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:49.536714    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:49.537178 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:49.536714    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:49.537178 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:49.536816    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98gq6" podUID="5b9b648c-4c7d-4f02-a6b3-c8e719f972b4"
Sep 18 06:22:49.537178 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:49.536714    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:49.537178 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:49.536889    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-n8fkx" podUID="1a0016b5-3d41-4c0c-afd6-bf60b5e69237"
Sep 18 06:22:49.537178 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:49.536964    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6z9c" podUID="bcf277d6-51cd-4bc8-ba3f-e6fcf272d363"
Sep 18 06:22:50.146987 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.146949    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret\") pod \"global-pull-secret-syncer-n8fkx\" (UID: \"1a0016b5-3d41-4c0c-afd6-bf60b5e69237\") " pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:50.147247 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:50.147206    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 06:22:50.147336 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:50.147296    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret podName:1a0016b5-3d41-4c0c-afd6-bf60b5e69237 nodeName:}" failed. No retries permitted until 2026-09-18 06:23:22.147274819 +0000 UTC m=+67.828802545 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret") pod "global-pull-secret-syncer-n8fkx" (UID: "1a0016b5-3d41-4c0c-afd6-bf60b5e69237") : object "kube-system"/"original-pull-secret" not registered
Sep 18 06:22:50.247911 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.247883    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs\") pod \"network-metrics-daemon-k6z9c\" (UID: \"bcf277d6-51cd-4bc8-ba3f-e6fcf272d363\") " pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:50.248099 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:50.248047    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 06:22:50.248166 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:50.248121    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs podName:bcf277d6-51cd-4bc8-ba3f-e6fcf272d363 nodeName:}" failed. No retries permitted until 2026-09-18 06:23:22.248103316 +0000 UTC m=+67.929631053 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs") pod "network-metrics-daemon-k6z9c" (UID: "bcf277d6-51cd-4bc8-ba3f-e6fcf272d363") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 06:22:50.348779 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.348707    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8p5b\" (UniqueName: \"kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b\") pod \"network-check-target-98gq6\" (UID: \"5b9b648c-4c7d-4f02-a6b3-c8e719f972b4\") " pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:50.348923 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:50.348863    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 06:22:50.348923 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:50.348882    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 06:22:50.348923 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:50.348892    2568 projected.go:196] Error preparing data for projected volume kube-api-access-p8p5b for pod openshift-network-diagnostics/network-check-target-98gq6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 06:22:50.349713 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:50.349323    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b podName:5b9b648c-4c7d-4f02-a6b3-c8e719f972b4 nodeName:}" failed. No retries permitted until 2026-09-18 06:23:22.349291354 +0000 UTC m=+68.030819130 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-p8p5b" (UniqueName: "kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b") pod "network-check-target-98gq6" (UID: "5b9b648c-4c7d-4f02-a6b3-c8e719f972b4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 06:22:50.476510 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.476474    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeReady"
Sep 18 06:22:50.476688 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.476612    2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 18 06:22:50.518075 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.518042    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jdjzn"]
Sep 18 06:22:50.539600 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.539569    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:22:50.542422 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.542178    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 18 06:22:50.542422 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.542200    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 18 06:22:50.542422 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.542201    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-sn4gt\""
Sep 18 06:22:50.543198 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.543176    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wz2zs"]
Sep 18 06:22:50.559519 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.559500    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jdjzn"]
Sep 18 06:22:50.559624 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.559530    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wz2zs"]
Sep 18 06:22:50.559670 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.559627    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wz2zs"
Sep 18 06:22:50.561972 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.561953    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 18 06:22:50.561972 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.561971    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2l9zb\""
Sep 18 06:22:50.562161 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.561998    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 18 06:22:50.562456 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.562439    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 18 06:22:50.624212 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.624143    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:50.650966 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.650936    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1f10a14-3b4a-4232-a2b0-5328c2384cb8-cert\") pod \"ingress-canary-wz2zs\" (UID: \"d1f10a14-3b4a-4232-a2b0-5328c2384cb8\") " pod="openshift-ingress-canary/ingress-canary-wz2zs"
Sep 18 06:22:50.651099 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.650979    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4a5817a8-34ca-4d89-8543-76631f2d2d62-config-volume\") pod \"dns-default-jdjzn\" (UID: \"4a5817a8-34ca-4d89-8543-76631f2d2d62\") " pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:22:50.651099 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.651018    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls\") pod \"dns-default-jdjzn\" (UID: \"4a5817a8-34ca-4d89-8543-76631f2d2d62\") " pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:22:50.651099 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.651042    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhfzw\" (UniqueName: \"kubernetes.io/projected/d1f10a14-3b4a-4232-a2b0-5328c2384cb8-kube-api-access-zhfzw\") pod \"ingress-canary-wz2zs\" (UID: \"d1f10a14-3b4a-4232-a2b0-5328c2384cb8\") " pod="openshift-ingress-canary/ingress-canary-wz2zs"
Sep 18 06:22:50.651099 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.651079    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5v72\" (UniqueName: \"kubernetes.io/projected/4a5817a8-34ca-4d89-8543-76631f2d2d62-kube-api-access-r5v72\") pod \"dns-default-jdjzn\" (UID: \"4a5817a8-34ca-4d89-8543-76631f2d2d62\") " pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:22:50.651299 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.651162    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a5817a8-34ca-4d89-8543-76631f2d2d62-tmp-dir\") pod \"dns-default-jdjzn\" (UID: \"4a5817a8-34ca-4d89-8543-76631f2d2d62\") " pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:22:50.678771 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.678731    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:50.752317 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.752287    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a5817a8-34ca-4d89-8543-76631f2d2d62-tmp-dir\") pod \"dns-default-jdjzn\" (UID: \"4a5817a8-34ca-4d89-8543-76631f2d2d62\") " pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:22:50.752484 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.752339    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1f10a14-3b4a-4232-a2b0-5328c2384cb8-cert\") pod \"ingress-canary-wz2zs\" (UID: \"d1f10a14-3b4a-4232-a2b0-5328c2384cb8\") " pod="openshift-ingress-canary/ingress-canary-wz2zs"
Sep 18 06:22:50.752484 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.752379    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4a5817a8-34ca-4d89-8543-76631f2d2d62-config-volume\") pod \"dns-default-jdjzn\" (UID: \"4a5817a8-34ca-4d89-8543-76631f2d2d62\") " pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:22:50.752484 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.752410    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls\") pod \"dns-default-jdjzn\" (UID: \"4a5817a8-34ca-4d89-8543-76631f2d2d62\") " pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:22:50.752484 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.752433    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhfzw\" (UniqueName: \"kubernetes.io/projected/d1f10a14-3b4a-4232-a2b0-5328c2384cb8-kube-api-access-zhfzw\") pod \"ingress-canary-wz2zs\" (UID: \"d1f10a14-3b4a-4232-a2b0-5328c2384cb8\") " pod="openshift-ingress-canary/ingress-canary-wz2zs"
Sep 18 06:22:50.752700 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.752498    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5v72\" (UniqueName: \"kubernetes.io/projected/4a5817a8-34ca-4d89-8543-76631f2d2d62-kube-api-access-r5v72\") pod \"dns-default-jdjzn\" (UID: \"4a5817a8-34ca-4d89-8543-76631f2d2d62\") " pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:22:50.752700 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:50.752524    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 06:22:50.752700 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:50.752602    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls podName:4a5817a8-34ca-4d89-8543-76631f2d2d62 nodeName:}" failed. No retries permitted until 2026-09-18 06:22:51.252580187 +0000 UTC m=+36.934107927 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls") pod "dns-default-jdjzn" (UID: "4a5817a8-34ca-4d89-8543-76631f2d2d62") : secret "dns-default-metrics-tls" not found
Sep 18 06:22:50.752700 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.752608    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a5817a8-34ca-4d89-8543-76631f2d2d62-tmp-dir\") pod \"dns-default-jdjzn\" (UID: \"4a5817a8-34ca-4d89-8543-76631f2d2d62\") " pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:22:50.753029 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.753001    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4a5817a8-34ca-4d89-8543-76631f2d2d62-config-volume\") pod \"dns-default-jdjzn\" (UID: \"4a5817a8-34ca-4d89-8543-76631f2d2d62\") " pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:22:50.757084 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.757059    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1f10a14-3b4a-4232-a2b0-5328c2384cb8-cert\") pod \"ingress-canary-wz2zs\" (UID: \"d1f10a14-3b4a-4232-a2b0-5328c2384cb8\") " pod="openshift-ingress-canary/ingress-canary-wz2zs"
Sep 18 06:22:50.761411 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.761379    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5v72\" (UniqueName: \"kubernetes.io/projected/4a5817a8-34ca-4d89-8543-76631f2d2d62-kube-api-access-r5v72\") pod \"dns-default-jdjzn\" (UID: \"4a5817a8-34ca-4d89-8543-76631f2d2d62\") " pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:22:50.761517 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.761459    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhfzw\" (UniqueName: \"kubernetes.io/projected/d1f10a14-3b4a-4232-a2b0-5328c2384cb8-kube-api-access-zhfzw\") pod \"ingress-canary-wz2zs\" (UID: \"d1f10a14-3b4a-4232-a2b0-5328c2384cb8\") " pod="openshift-ingress-canary/ingress-canary-wz2zs"
Sep 18 06:22:50.869967 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:50.869939    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wz2zs"
Sep 18 06:22:51.254815 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:51.254720    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls\") pod \"dns-default-jdjzn\" (UID: \"4a5817a8-34ca-4d89-8543-76631f2d2d62\") " pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:22:51.254970 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:51.254859    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 06:22:51.254970 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:51.254923    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls podName:4a5817a8-34ca-4d89-8543-76631f2d2d62 nodeName:}" failed. No retries permitted until 2026-09-18 06:22:52.254902665 +0000 UTC m=+37.936430405 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls") pod "dns-default-jdjzn" (UID: "4a5817a8-34ca-4d89-8543-76631f2d2d62") : secret "dns-default-metrics-tls" not found
Sep 18 06:22:51.428846 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:51.428813    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:51.528903 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:51.528814    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:51.536569 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:51.536546    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:22:51.536719 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:51.536549    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:22:51.536719 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:51.536685    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:22:51.539282 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:51.539261    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 06:22:51.540384 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:51.540362    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 06:22:51.540876 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:51.540433    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 06:22:51.540876 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:51.540478    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kj54z\""
Sep 18 06:22:51.540876 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:51.540515    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 06:22:51.540876 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:51.540638    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fg7cb\""
Sep 18 06:22:52.171482 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:52.171456    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wz2zs"]
Sep 18 06:22:52.191624 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:22:52.191598    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1f10a14_3b4a_4232_a2b0_5328c2384cb8.slice/crio-1aed8f775343f1f4c6d1ed8ad1cbf084989ddb4704f79f577e53012e500d50cc WatchSource:0}: Error finding container 1aed8f775343f1f4c6d1ed8ad1cbf084989ddb4704f79f577e53012e500d50cc: Status 404 returned error can't find the container with id 1aed8f775343f1f4c6d1ed8ad1cbf084989ddb4704f79f577e53012e500d50cc
Sep 18 06:22:52.263862 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:52.263838    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls\") pod \"dns-default-jdjzn\" (UID: \"4a5817a8-34ca-4d89-8543-76631f2d2d62\") " pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:22:52.264019 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:52.264002    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 06:22:52.264082 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:52.264060    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls podName:4a5817a8-34ca-4d89-8543-76631f2d2d62 nodeName:}" failed. No retries permitted until 2026-09-18 06:22:54.264043414 +0000 UTC m=+39.945571153 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls") pod "dns-default-jdjzn" (UID: "4a5817a8-34ca-4d89-8543-76631f2d2d62") : secret "dns-default-metrics-tls" not found
Sep 18 06:22:52.870660 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:52.870339    2568 generic.go:356] "Generic (PLEG): container finished" podID="688032c8-c1d8-4121-8ee0-c224615cc527" containerID="00c213202abe265a5400fabfecf40655e5df6e5c391838f7129c3f15688d008c" exitCode=0
Sep 18 06:22:52.870660 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:52.870413    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z26mm" event={"ID":"688032c8-c1d8-4121-8ee0-c224615cc527","Type":"ContainerDied","Data":"00c213202abe265a5400fabfecf40655e5df6e5c391838f7129c3f15688d008c"}
Sep 18 06:22:52.872114 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:52.872085    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wz2zs" event={"ID":"d1f10a14-3b4a-4232-a2b0-5328c2384cb8","Type":"ContainerStarted","Data":"1aed8f775343f1f4c6d1ed8ad1cbf084989ddb4704f79f577e53012e500d50cc"}
Sep 18 06:22:53.878098 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:53.878066    2568 generic.go:356] "Generic (PLEG): container finished" podID="688032c8-c1d8-4121-8ee0-c224615cc527" containerID="163858b9b1f567b4e9b27fc7d56601798375e606cc1bd04acc5e6375b909656d" exitCode=0
Sep 18 06:22:53.878549 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:53.878148    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z26mm" event={"ID":"688032c8-c1d8-4121-8ee0-c224615cc527","Type":"ContainerDied","Data":"163858b9b1f567b4e9b27fc7d56601798375e606cc1bd04acc5e6375b909656d"}
Sep 18 06:22:54.278960 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:54.278922    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls\") pod \"dns-default-jdjzn\" (UID: \"4a5817a8-34ca-4d89-8543-76631f2d2d62\") " pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:22:54.279132 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:54.279023    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 06:22:54.279132 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:54.279098    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls podName:4a5817a8-34ca-4d89-8543-76631f2d2d62 nodeName:}" failed. No retries permitted until 2026-09-18 06:22:58.27907785 +0000 UTC m=+43.960605595 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls") pod "dns-default-jdjzn" (UID: "4a5817a8-34ca-4d89-8543-76631f2d2d62") : secret "dns-default-metrics-tls" not found
Sep 18 06:22:54.752056 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:54.752027    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:54.803664 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:54.803600    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:54.883812 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:54.883776    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-z26mm" event={"ID":"688032c8-c1d8-4121-8ee0-c224615cc527","Type":"ContainerStarted","Data":"02f79353c989d8fd2dedcf47541ef4e134e616014669acb40405cd826cdcee76"}
Sep 18 06:22:54.906248 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:54.906192    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-z26mm" podStartSLOduration=10.048557075 podStartE2EDuration="38.906177461s" totalImagesPullingTime="28.857620386s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 06:22:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:22:54.90434156 +0000 UTC m=+40.585869305" watchObservedRunningTime="2026-09-18 06:22:54.906177461 +0000 UTC m=+40.587705205"
Sep 18 06:22:55.125302 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:55.125185    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:55.222870 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:55.222838    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:55.890776 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:55.890739    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wz2zs" event={"ID":"d1f10a14-3b4a-4232-a2b0-5328c2384cb8","Type":"ContainerStarted","Data":"36986b5644c3c0d34d284f83fac4270af05485140d1fd019ed12cc71f7ebd3b7"}
Sep 18 06:22:57.730231 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:57.730185    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:57.780831 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:57.780804    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:57.963366 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:57.963337    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:58.012375 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:58.012309    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:22:58.304927 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:22:58.304831    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls\") pod \"dns-default-jdjzn\" (UID: \"4a5817a8-34ca-4d89-8543-76631f2d2d62\") " pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:22:58.305091 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:58.304969    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 06:22:58.305091 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:22:58.305040    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls podName:4a5817a8-34ca-4d89-8543-76631f2d2d62 nodeName:}" failed. No retries permitted until 2026-09-18 06:23:06.305018575 +0000 UTC m=+51.986546311 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls") pod "dns-default-jdjzn" (UID: "4a5817a8-34ca-4d89-8543-76631f2d2d62") : secret "dns-default-metrics-tls" not found
Sep 18 06:23:00.341422 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:00.341381    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:23:00.392871 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:00.392843    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:23:02.701924 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:02.701891    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:23:02.750391 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:02.750358    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:23:06.357990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:06.357946    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls\") pod \"dns-default-jdjzn\" (UID: \"4a5817a8-34ca-4d89-8543-76631f2d2d62\") " pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:23:06.358389 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:23:06.358097    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 06:23:06.358389 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:23:06.358172    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls podName:4a5817a8-34ca-4d89-8543-76631f2d2d62 nodeName:}" failed. No retries permitted until 2026-09-18 06:23:22.358156939 +0000 UTC m=+68.039684674 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls") pod "dns-default-jdjzn" (UID: "4a5817a8-34ca-4d89-8543-76631f2d2d62") : secret "dns-default-metrics-tls" not found
Sep 18 06:23:15.869634 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:15.869602    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-l6kp8"
Sep 18 06:23:15.897566 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:15.897509    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wz2zs" podStartSLOduration=23.195266784 podStartE2EDuration="25.897489208s" totalImagesPullingTime="2.702222424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:22:55.904984684 +0000 UTC m=+41.586512474" watchObservedRunningTime="2026-09-18 06:23:15.897489208 +0000 UTC m=+61.579016950"
Sep 18 06:23:16.018559 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:16.018527    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:23:16.074835 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:16.074805    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:23:19.709186 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:19.709142    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:23:19.760737 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:19.760702    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:23:22.159258 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:22.159212    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret\") pod \"global-pull-secret-syncer-n8fkx\" (UID: \"1a0016b5-3d41-4c0c-afd6-bf60b5e69237\") " pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:23:22.165007 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:22.164989    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 06:23:22.173637 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:22.173607    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a0016b5-3d41-4c0c-afd6-bf60b5e69237-original-pull-secret\") pod \"global-pull-secret-syncer-n8fkx\" (UID: \"1a0016b5-3d41-4c0c-afd6-bf60b5e69237\") " pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:23:22.260424 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:22.260390    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs\") pod \"network-metrics-daemon-k6z9c\" (UID: \"bcf277d6-51cd-4bc8-ba3f-e6fcf272d363\") " pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:23:22.262847 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:22.262831    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 06:23:22.270788 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:23:22.270773    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 18 06:23:22.270839 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:23:22.270820    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs podName:bcf277d6-51cd-4bc8-ba3f-e6fcf272d363 nodeName:}" failed. No retries permitted until 2026-09-18 06:24:26.270805201 +0000 UTC m=+131.952332922 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs") pod "network-metrics-daemon-k6z9c" (UID: "bcf277d6-51cd-4bc8-ba3f-e6fcf272d363") : secret "metrics-daemon-secret" not found
Sep 18 06:23:22.361652 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:22.361624    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8p5b\" (UniqueName: \"kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b\") pod \"network-check-target-98gq6\" (UID: \"5b9b648c-4c7d-4f02-a6b3-c8e719f972b4\") " pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:23:22.361652 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:22.361655    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls\") pod \"dns-default-jdjzn\" (UID: \"4a5817a8-34ca-4d89-8543-76631f2d2d62\") " pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:23:22.361775 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:23:22.361744    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 06:23:22.361828 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:23:22.361788    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls podName:4a5817a8-34ca-4d89-8543-76631f2d2d62 nodeName:}" failed. No retries permitted until 2026-09-18 06:23:54.3617769 +0000 UTC m=+100.043304622 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls") pod "dns-default-jdjzn" (UID: "4a5817a8-34ca-4d89-8543-76631f2d2d62") : secret "dns-default-metrics-tls" not found
Sep 18 06:23:22.364578 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:22.364561    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 06:23:22.374862 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:22.374844    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 06:23:22.386104 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:22.386079    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8p5b\" (UniqueName: \"kubernetes.io/projected/5b9b648c-4c7d-4f02-a6b3-c8e719f972b4-kube-api-access-p8p5b\") pod \"network-check-target-98gq6\" (UID: \"5b9b648c-4c7d-4f02-a6b3-c8e719f972b4\") " pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:23:22.457069 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:22.457010    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kj54z\""
Sep 18 06:23:22.459839 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:22.459827    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-n8fkx"
Sep 18 06:23:22.465293 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:22.465277    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:23:22.731164 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:22.731096    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-n8fkx"]
Sep 18 06:23:22.734640 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:22.734612    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-98gq6"]
Sep 18 06:23:22.734864 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:23:22.734834    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a0016b5_3d41_4c0c_afd6_bf60b5e69237.slice/crio-7ce83675c1acceb922ae23573c5f8d906c0728bbbd3766df05effe0d678e7634 WatchSource:0}: Error finding container 7ce83675c1acceb922ae23573c5f8d906c0728bbbd3766df05effe0d678e7634: Status 404 returned error can't find the container with id 7ce83675c1acceb922ae23573c5f8d906c0728bbbd3766df05effe0d678e7634
Sep 18 06:23:22.951835 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:22.951800    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-98gq6" event={"ID":"5b9b648c-4c7d-4f02-a6b3-c8e719f972b4","Type":"ContainerStarted","Data":"12a264ed2c8895022d223a823d4f736274c9e5d76c49caa278d3489a9953bdf9"}
Sep 18 06:23:22.952897 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:22.952871    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n8fkx" event={"ID":"1a0016b5-3d41-4c0c-afd6-bf60b5e69237","Type":"ContainerStarted","Data":"7ce83675c1acceb922ae23573c5f8d906c0728bbbd3766df05effe0d678e7634"}
Sep 18 06:23:27.967705 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:27.967670    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-98gq6" event={"ID":"5b9b648c-4c7d-4f02-a6b3-c8e719f972b4","Type":"ContainerStarted","Data":"778236f0559d8e944751f85cef6927ed4502ed04575202841c0b3b2068b16e66"}
Sep 18 06:23:27.968137 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:27.967762    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:23:27.969189 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:27.969169    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-n8fkx" event={"ID":"1a0016b5-3d41-4c0c-afd6-bf60b5e69237","Type":"ContainerStarted","Data":"9645070b0cb3c7af1be168e3ed118f879d662f1673c559091a5fab8dd1e9374e"}
Sep 18 06:23:27.981861 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:27.981821    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-98gq6" podStartSLOduration=67.46155173 podStartE2EDuration="1m11.981806265s" totalImagesPullingTime="4.520254535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:22:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:23:27.98125043 +0000 UTC m=+73.662778170" watchObservedRunningTime="2026-09-18 06:23:27.981806265 +0000 UTC m=+73.663334010"
Sep 18 06:23:33.915771 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:33.915710    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-n8fkx" podStartSLOduration=73.383853869 podStartE2EDuration="1m17.915690957s" totalImagesPullingTime="4.531837088s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 06:22:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:23:27.995710717 +0000 UTC m=+73.677238461" watchObservedRunningTime="2026-09-18 06:23:33.915690957 +0000 UTC m=+79.597218722"
Sep 18 06:23:33.916191 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:33.915990    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"]
Sep 18 06:23:33.917985 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:33.917966    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"
Sep 18 06:23:33.920407 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:33.920383    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 18 06:23:33.920507 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:33.920406    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 18 06:23:33.920580 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:33.920550    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 18 06:23:33.921280 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:33.921265    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 18 06:23:33.921361 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:33.921301    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 18 06:23:33.921408 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:33.921367    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 18 06:23:33.921458 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:33.921428    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 18 06:23:33.928771 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:33.928752    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"]
Sep 18 06:23:33.947040 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:33.947005    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/743c3866-3864-4879-8784-1dbf1a28414f-ca\") pod \"cluster-proxy-proxy-agent-5645ddf6bc-kbfvl\" (UID: \"743c3866-3864-4879-8784-1dbf1a28414f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"
Sep 18 06:23:33.947040 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:33.947043    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/743c3866-3864-4879-8784-1dbf1a28414f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5645ddf6bc-kbfvl\" (UID: \"743c3866-3864-4879-8784-1dbf1a28414f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"
Sep 18 06:23:33.947174 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:33.947063    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/743c3866-3864-4879-8784-1dbf1a28414f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5645ddf6bc-kbfvl\" (UID: \"743c3866-3864-4879-8784-1dbf1a28414f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"
Sep 18 06:23:33.947174 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:33.947136    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcpcr\" (UniqueName: \"kubernetes.io/projected/743c3866-3864-4879-8784-1dbf1a28414f-kube-api-access-hcpcr\") pod \"cluster-proxy-proxy-agent-5645ddf6bc-kbfvl\" (UID: \"743c3866-3864-4879-8784-1dbf1a28414f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"
Sep 18 06:23:33.947260 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:33.947199    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/743c3866-3864-4879-8784-1dbf1a28414f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5645ddf6bc-kbfvl\" (UID: \"743c3866-3864-4879-8784-1dbf1a28414f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"
Sep 18 06:23:33.947260 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:33.947250    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/743c3866-3864-4879-8784-1dbf1a28414f-hub\") pod \"cluster-proxy-proxy-agent-5645ddf6bc-kbfvl\" (UID: \"743c3866-3864-4879-8784-1dbf1a28414f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"
Sep 18 06:23:34.048560 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:34.048517    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/743c3866-3864-4879-8784-1dbf1a28414f-hub\") pod \"cluster-proxy-proxy-agent-5645ddf6bc-kbfvl\" (UID: \"743c3866-3864-4879-8784-1dbf1a28414f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"
Sep 18 06:23:34.048560 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:34.048567    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/743c3866-3864-4879-8784-1dbf1a28414f-ca\") pod \"cluster-proxy-proxy-agent-5645ddf6bc-kbfvl\" (UID: \"743c3866-3864-4879-8784-1dbf1a28414f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"
Sep 18 06:23:34.048763 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:34.048586    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/743c3866-3864-4879-8784-1dbf1a28414f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5645ddf6bc-kbfvl\" (UID: \"743c3866-3864-4879-8784-1dbf1a28414f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"
Sep 18 06:23:34.048763 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:34.048660    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/743c3866-3864-4879-8784-1dbf1a28414f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5645ddf6bc-kbfvl\" (UID: \"743c3866-3864-4879-8784-1dbf1a28414f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"
Sep 18 06:23:34.048763 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:34.048719    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcpcr\" (UniqueName: \"kubernetes.io/projected/743c3866-3864-4879-8784-1dbf1a28414f-kube-api-access-hcpcr\") pod \"cluster-proxy-proxy-agent-5645ddf6bc-kbfvl\" (UID: \"743c3866-3864-4879-8784-1dbf1a28414f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"
Sep 18 06:23:34.048868 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:34.048800    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/743c3866-3864-4879-8784-1dbf1a28414f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5645ddf6bc-kbfvl\" (UID: \"743c3866-3864-4879-8784-1dbf1a28414f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"
Sep 18 06:23:34.049388 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:34.049362    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/743c3866-3864-4879-8784-1dbf1a28414f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5645ddf6bc-kbfvl\" (UID: \"743c3866-3864-4879-8784-1dbf1a28414f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"
Sep 18 06:23:34.051271 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:34.051245    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/743c3866-3864-4879-8784-1dbf1a28414f-ca\") pod \"cluster-proxy-proxy-agent-5645ddf6bc-kbfvl\" (UID: \"743c3866-3864-4879-8784-1dbf1a28414f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"
Sep 18 06:23:34.051379 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:34.051295    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/743c3866-3864-4879-8784-1dbf1a28414f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5645ddf6bc-kbfvl\" (UID: \"743c3866-3864-4879-8784-1dbf1a28414f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"
Sep 18 06:23:34.051379 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:34.051322    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/743c3866-3864-4879-8784-1dbf1a28414f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5645ddf6bc-kbfvl\" (UID: \"743c3866-3864-4879-8784-1dbf1a28414f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"
Sep 18 06:23:34.051379 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:34.051371    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/743c3866-3864-4879-8784-1dbf1a28414f-hub\") pod \"cluster-proxy-proxy-agent-5645ddf6bc-kbfvl\" (UID: \"743c3866-3864-4879-8784-1dbf1a28414f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"
Sep 18 06:23:34.055419 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:34.055402    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcpcr\" (UniqueName: \"kubernetes.io/projected/743c3866-3864-4879-8784-1dbf1a28414f-kube-api-access-hcpcr\") pod \"cluster-proxy-proxy-agent-5645ddf6bc-kbfvl\" (UID: \"743c3866-3864-4879-8784-1dbf1a28414f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"
Sep 18 06:23:34.240882 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:34.240779    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"
Sep 18 06:23:34.508461 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:34.508391    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"]
Sep 18 06:23:34.511644 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:23:34.511618    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod743c3866_3864_4879_8784_1dbf1a28414f.slice/crio-06c4d64798bcae154f5e148dfae9f8f337ea5e322b7c47bb1308a4e7a0eaad75 WatchSource:0}: Error finding container 06c4d64798bcae154f5e148dfae9f8f337ea5e322b7c47bb1308a4e7a0eaad75: Status 404 returned error can't find the container with id 06c4d64798bcae154f5e148dfae9f8f337ea5e322b7c47bb1308a4e7a0eaad75
Sep 18 06:23:34.990807 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:34.990773    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl" event={"ID":"743c3866-3864-4879-8784-1dbf1a28414f","Type":"ContainerStarted","Data":"06c4d64798bcae154f5e148dfae9f8f337ea5e322b7c47bb1308a4e7a0eaad75"}
Sep 18 06:23:39.005493 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:39.005454    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl" event={"ID":"743c3866-3864-4879-8784-1dbf1a28414f","Type":"ContainerStarted","Data":"bd27031aa555b5509eb4d33346eafc9e420e5cf6e1f1ecfba628656bd973c97d"}
Sep 18 06:23:39.005879 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:39.005509    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl" event={"ID":"743c3866-3864-4879-8784-1dbf1a28414f","Type":"ContainerStarted","Data":"fec9279d46a3544da2e14b3c90f3bb9d6038a6f9552afaf87ba795cb44756cbd"}
Sep 18 06:23:39.005879 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:39.005526    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl" event={"ID":"743c3866-3864-4879-8784-1dbf1a28414f","Type":"ContainerStarted","Data":"fd88568ab5de43adaa1ccdf56dbb1a35ca2e26720baff57a9d11cea7c0bf501d"}
Sep 18 06:23:39.022605 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:39.022560    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl" podStartSLOduration=2.151839446 podStartE2EDuration="6.022548033s" totalImagesPullingTime="3.870708587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:23:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:23:39.021293475 +0000 UTC m=+84.702821220" watchObservedRunningTime="2026-09-18 06:23:39.022548033 +0000 UTC m=+84.704075777"
Sep 18 06:23:42.775950 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:42.775912    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:23:42.824184 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:42.824147    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:23:54.397417 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:54.397383    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls\") pod \"dns-default-jdjzn\" (UID: \"4a5817a8-34ca-4d89-8543-76631f2d2d62\") " pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:23:54.397828 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:23:54.397511    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 06:23:54.397828 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:23:54.397566    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls podName:4a5817a8-34ca-4d89-8543-76631f2d2d62 nodeName:}" failed. No retries permitted until 2026-09-18 06:24:58.3975506 +0000 UTC m=+164.079078341 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls") pod "dns-default-jdjzn" (UID: "4a5817a8-34ca-4d89-8543-76631f2d2d62") : secret "dns-default-metrics-tls" not found
Sep 18 06:23:58.974548 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:23:58.974519    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-98gq6"
Sep 18 06:24:26.307087 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:24:26.307042    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs\") pod \"network-metrics-daemon-k6z9c\" (UID: \"bcf277d6-51cd-4bc8-ba3f-e6fcf272d363\") " pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:24:26.307598 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:24:26.307197    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 18 06:24:26.307598 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:24:26.307294    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs podName:bcf277d6-51cd-4bc8-ba3f-e6fcf272d363 nodeName:}" failed. No retries permitted until 2026-09-18 06:26:28.307274661 +0000 UTC m=+253.988802403 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs") pod "network-metrics-daemon-k6z9c" (UID: "bcf277d6-51cd-4bc8-ba3f-e6fcf272d363") : secret "metrics-daemon-secret" not found
Sep 18 06:24:53.551234 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:24:53.551164    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-jdjzn" podUID="4a5817a8-34ca-4d89-8543-76631f2d2d62"
Sep 18 06:24:54.214704 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:24:54.214677    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:24:54.548328 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:24:54.548237    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-k6z9c" podUID="bcf277d6-51cd-4bc8-ba3f-e6fcf272d363"
Sep 18 06:24:58.423327 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:24:58.423235    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls\") pod \"dns-default-jdjzn\" (UID: \"4a5817a8-34ca-4d89-8543-76631f2d2d62\") " pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:24:58.423695 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:24:58.423381    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 06:24:58.423695 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:24:58.423450    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls podName:4a5817a8-34ca-4d89-8543-76631f2d2d62 nodeName:}" failed. No retries permitted until 2026-09-18 06:27:00.423430714 +0000 UTC m=+286.104958456 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls") pod "dns-default-jdjzn" (UID: "4a5817a8-34ca-4d89-8543-76631f2d2d62") : secret "dns-default-metrics-tls" not found
Sep 18 06:25:09.536632 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:25:09.536596    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:25:58.442016 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:25:58.441955    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:27.008194 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:27.008165    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-tgxrx"]
Sep 18 06:26:27.010842 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:27.010828    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-tgxrx"
Sep 18 06:26:27.013383 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:27.013368    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-jdrmj\""
Sep 18 06:26:27.020807 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:27.020785    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-tgxrx"]
Sep 18 06:26:27.083573 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:27.083512    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grtzv\" (UniqueName: \"kubernetes.io/projected/5b75c191-f17b-4e07-a2e8-5042d74062b9-kube-api-access-grtzv\") pod \"network-check-source-7cbc45fddc-tgxrx\" (UID: \"5b75c191-f17b-4e07-a2e8-5042d74062b9\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-tgxrx"
Sep 18 06:26:27.122200 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:27.122159    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:27.184435 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:27.184409    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grtzv\" (UniqueName: \"kubernetes.io/projected/5b75c191-f17b-4e07-a2e8-5042d74062b9-kube-api-access-grtzv\") pod \"network-check-source-7cbc45fddc-tgxrx\" (UID: \"5b75c191-f17b-4e07-a2e8-5042d74062b9\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-tgxrx"
Sep 18 06:26:27.191757 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:27.191730    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grtzv\" (UniqueName: \"kubernetes.io/projected/5b75c191-f17b-4e07-a2e8-5042d74062b9-kube-api-access-grtzv\") pod \"network-check-source-7cbc45fddc-tgxrx\" (UID: \"5b75c191-f17b-4e07-a2e8-5042d74062b9\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-tgxrx"
Sep 18 06:26:27.222184 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:27.222159    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:27.319660 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:27.319596    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-tgxrx"
Sep 18 06:26:27.583781 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:27.583713    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-tgxrx"]
Sep 18 06:26:27.586402 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:26:27.586378    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b75c191_f17b_4e07_a2e8_5042d74062b9.slice/crio-493b8305cc610235ebeb0ff0979d004a07b0b60cc141beac255d29f7dfe3ad13 WatchSource:0}: Error finding container 493b8305cc610235ebeb0ff0979d004a07b0b60cc141beac255d29f7dfe3ad13: Status 404 returned error can't find the container with id 493b8305cc610235ebeb0ff0979d004a07b0b60cc141beac255d29f7dfe3ad13
Sep 18 06:26:28.392888 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:28.392850    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs\") pod \"network-metrics-daemon-k6z9c\" (UID: \"bcf277d6-51cd-4bc8-ba3f-e6fcf272d363\") " pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:26:28.393362 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:28.392964    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 18 06:26:28.393362 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:28.393022    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs podName:bcf277d6-51cd-4bc8-ba3f-e6fcf272d363 nodeName:}" failed. No retries permitted until 2026-09-18 06:28:30.393005215 +0000 UTC m=+376.074532941 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs") pod "network-metrics-daemon-k6z9c" (UID: "bcf277d6-51cd-4bc8-ba3f-e6fcf272d363") : secret "metrics-daemon-secret" not found
Sep 18 06:26:28.466152 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:28.466121    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-tgxrx" event={"ID":"5b75c191-f17b-4e07-a2e8-5042d74062b9","Type":"ContainerStarted","Data":"c03a0b75894f4c128f2e4f75abf4e9771ef4f9c725bbc78b2f2d1f2548ab7c87"}
Sep 18 06:26:28.466345 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:28.466168    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-tgxrx" event={"ID":"5b75c191-f17b-4e07-a2e8-5042d74062b9","Type":"ContainerStarted","Data":"493b8305cc610235ebeb0ff0979d004a07b0b60cc141beac255d29f7dfe3ad13"}
Sep 18 06:26:28.481009 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:28.480964    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-tgxrx" podStartSLOduration=2.480948144 podStartE2EDuration="2.480948144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:26:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:26:28.480259189 +0000 UTC m=+254.161786933" watchObservedRunningTime="2026-09-18 06:26:28.480948144 +0000 UTC m=+254.162475892"
Sep 18 06:26:29.252825 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:29.252794    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:29.353646 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:29.353612    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:31.494883 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:31.494857    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:31.587180 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:31.587152    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:31.690571 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:31.690545    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:31.793322 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:31.793268    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:31.953773 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:31.953747    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rqpls"]
Sep 18 06:26:31.956808 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:31.956791    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:31.958983 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:31.958965    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 18 06:26:31.959156 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:31.959141    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 18 06:26:31.959363 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:31.959351    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 18 06:26:31.960175 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:31.960162    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6n6ts\""
Sep 18 06:26:31.960257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:31.960231    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 18 06:26:31.967966 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:31.967941    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rqpls"]
Sep 18 06:26:31.975167 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:31.975149    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:32.022096 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:32.022070    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f412820-0d80-43dc-94bd-fd4b5a773fde-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:32.022262 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:32.022160    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9f412820-0d80-43dc-94bd-fd4b5a773fde-crio-socket\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:32.022262 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:32.022188    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f412820-0d80-43dc-94bd-fd4b5a773fde-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:32.022262 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:32.022206    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mttxt\" (UniqueName: \"kubernetes.io/projected/9f412820-0d80-43dc-94bd-fd4b5a773fde-kube-api-access-mttxt\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:32.022262 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:32.022248    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f412820-0d80-43dc-94bd-fd4b5a773fde-data-volume\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:32.022479 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:32.022456    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:32.123348 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:32.123266    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9f412820-0d80-43dc-94bd-fd4b5a773fde-crio-socket\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:32.123348 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:32.123308    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f412820-0d80-43dc-94bd-fd4b5a773fde-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:32.123348 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:32.123327    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mttxt\" (UniqueName: \"kubernetes.io/projected/9f412820-0d80-43dc-94bd-fd4b5a773fde-kube-api-access-mttxt\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:32.123648 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:32.123385    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 06:26:32.123648 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:32.123415    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9f412820-0d80-43dc-94bd-fd4b5a773fde-crio-socket\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:32.123648 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:32.123436    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f412820-0d80-43dc-94bd-fd4b5a773fde-data-volume\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:32.123648 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:32.123458    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9f412820-0d80-43dc-94bd-fd4b5a773fde-kube-rbac-proxy-cm podName:9f412820-0d80-43dc-94bd-fd4b5a773fde nodeName:}" failed. No retries permitted until 2026-09-18 06:26:32.62343914 +0000 UTC m=+258.304966869 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9f412820-0d80-43dc-94bd-fd4b5a773fde-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rqpls" (UID: "9f412820-0d80-43dc-94bd-fd4b5a773fde") : configmap "kube-rbac-proxy" not found
Sep 18 06:26:32.123648 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:32.123578    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f412820-0d80-43dc-94bd-fd4b5a773fde-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:32.123821 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:32.123712    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 06:26:32.123821 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:32.123719    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9f412820-0d80-43dc-94bd-fd4b5a773fde-data-volume\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:32.123821 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:32.123774    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f412820-0d80-43dc-94bd-fd4b5a773fde-insights-runtime-extractor-tls podName:9f412820-0d80-43dc-94bd-fd4b5a773fde nodeName:}" failed. No retries permitted until 2026-09-18 06:26:32.623760844 +0000 UTC m=+258.305288570 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9f412820-0d80-43dc-94bd-fd4b5a773fde-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rqpls" (UID: "9f412820-0d80-43dc-94bd-fd4b5a773fde") : secret "insights-runtime-extractor-tls" not found
Sep 18 06:26:32.132559 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:32.132533    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mttxt\" (UniqueName: \"kubernetes.io/projected/9f412820-0d80-43dc-94bd-fd4b5a773fde-kube-api-access-mttxt\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:32.272086 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:32.272062    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:32.375015 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:32.374948    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:32.627812 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:32.627735    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f412820-0d80-43dc-94bd-fd4b5a773fde-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:32.627812 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:32.627794    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f412820-0d80-43dc-94bd-fd4b5a773fde-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:32.628212 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:32.627859    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 06:26:32.628212 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:32.627882    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 06:26:32.628212 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:32.627944    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9f412820-0d80-43dc-94bd-fd4b5a773fde-kube-rbac-proxy-cm podName:9f412820-0d80-43dc-94bd-fd4b5a773fde nodeName:}" failed. No retries permitted until 2026-09-18 06:26:33.627924705 +0000 UTC m=+259.309452431 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9f412820-0d80-43dc-94bd-fd4b5a773fde-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rqpls" (UID: "9f412820-0d80-43dc-94bd-fd4b5a773fde") : configmap "kube-rbac-proxy" not found
Sep 18 06:26:32.628212 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:32.627963    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f412820-0d80-43dc-94bd-fd4b5a773fde-insights-runtime-extractor-tls podName:9f412820-0d80-43dc-94bd-fd4b5a773fde nodeName:}" failed. No retries permitted until 2026-09-18 06:26:33.627954375 +0000 UTC m=+259.309482102 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9f412820-0d80-43dc-94bd-fd4b5a773fde-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rqpls" (UID: "9f412820-0d80-43dc-94bd-fd4b5a773fde") : secret "insights-runtime-extractor-tls" not found
Sep 18 06:26:33.633319 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:33.633283    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f412820-0d80-43dc-94bd-fd4b5a773fde-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:33.633712 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:33.633336    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f412820-0d80-43dc-94bd-fd4b5a773fde-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:33.633712 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:33.633416    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 06:26:33.633712 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:33.633424    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 06:26:33.633712 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:33.633466    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9f412820-0d80-43dc-94bd-fd4b5a773fde-kube-rbac-proxy-cm podName:9f412820-0d80-43dc-94bd-fd4b5a773fde nodeName:}" failed. No retries permitted until 2026-09-18 06:26:35.633452953 +0000 UTC m=+261.314980675 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9f412820-0d80-43dc-94bd-fd4b5a773fde-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rqpls" (UID: "9f412820-0d80-43dc-94bd-fd4b5a773fde") : configmap "kube-rbac-proxy" not found
Sep 18 06:26:33.633712 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:33.633478    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f412820-0d80-43dc-94bd-fd4b5a773fde-insights-runtime-extractor-tls podName:9f412820-0d80-43dc-94bd-fd4b5a773fde nodeName:}" failed. No retries permitted until 2026-09-18 06:26:35.633472509 +0000 UTC m=+261.315000231 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9f412820-0d80-43dc-94bd-fd4b5a773fde-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rqpls" (UID: "9f412820-0d80-43dc-94bd-fd4b5a773fde") : secret "insights-runtime-extractor-tls" not found
Sep 18 06:26:33.865604 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:33.865578    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-wmngx"]
Sep 18 06:26:33.868448 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:33.868434    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-wmngx"
Sep 18 06:26:33.870904 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:33.870885    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 18 06:26:33.871024 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:33.870885    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 18 06:26:33.871800 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:33.871779    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 18 06:26:33.871951 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:33.871846    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-f22th\""
Sep 18 06:26:33.871951 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:33.871892    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 18 06:26:33.877277 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:33.877257    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-wmngx"]
Sep 18 06:26:33.915179 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:33.915157    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:33.935517 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:33.935497    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq4z2\" (UniqueName: \"kubernetes.io/projected/f09da463-3ce0-459c-b52c-6e2fea2d1ed8-kube-api-access-wq4z2\") pod \"service-ca-55f6fc7b86-wmngx\" (UID: \"f09da463-3ce0-459c-b52c-6e2fea2d1ed8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-wmngx"
Sep 18 06:26:33.935581 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:33.935557    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f09da463-3ce0-459c-b52c-6e2fea2d1ed8-signing-key\") pod \"service-ca-55f6fc7b86-wmngx\" (UID: \"f09da463-3ce0-459c-b52c-6e2fea2d1ed8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-wmngx"
Sep 18 06:26:33.935641 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:33.935621    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f09da463-3ce0-459c-b52c-6e2fea2d1ed8-signing-cabundle\") pod \"service-ca-55f6fc7b86-wmngx\" (UID: \"f09da463-3ce0-459c-b52c-6e2fea2d1ed8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-wmngx"
Sep 18 06:26:33.973008 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:33.972985    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:34.036248 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:34.036212    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f09da463-3ce0-459c-b52c-6e2fea2d1ed8-signing-key\") pod \"service-ca-55f6fc7b86-wmngx\" (UID: \"f09da463-3ce0-459c-b52c-6e2fea2d1ed8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-wmngx"
Sep 18 06:26:34.036328 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:34.036271    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f09da463-3ce0-459c-b52c-6e2fea2d1ed8-signing-cabundle\") pod \"service-ca-55f6fc7b86-wmngx\" (UID: \"f09da463-3ce0-459c-b52c-6e2fea2d1ed8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-wmngx"
Sep 18 06:26:34.036328 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:34.036311    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq4z2\" (UniqueName: \"kubernetes.io/projected/f09da463-3ce0-459c-b52c-6e2fea2d1ed8-kube-api-access-wq4z2\") pod \"service-ca-55f6fc7b86-wmngx\" (UID: \"f09da463-3ce0-459c-b52c-6e2fea2d1ed8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-wmngx"
Sep 18 06:26:34.036966 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:34.036938    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f09da463-3ce0-459c-b52c-6e2fea2d1ed8-signing-cabundle\") pod \"service-ca-55f6fc7b86-wmngx\" (UID: \"f09da463-3ce0-459c-b52c-6e2fea2d1ed8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-wmngx"
Sep 18 06:26:34.038590 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:34.038574    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f09da463-3ce0-459c-b52c-6e2fea2d1ed8-signing-key\") pod \"service-ca-55f6fc7b86-wmngx\" (UID: \"f09da463-3ce0-459c-b52c-6e2fea2d1ed8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-wmngx"
Sep 18 06:26:34.043690 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:34.043666    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq4z2\" (UniqueName: \"kubernetes.io/projected/f09da463-3ce0-459c-b52c-6e2fea2d1ed8-kube-api-access-wq4z2\") pod \"service-ca-55f6fc7b86-wmngx\" (UID: \"f09da463-3ce0-459c-b52c-6e2fea2d1ed8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-wmngx"
Sep 18 06:26:34.177544 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:34.177446    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-wmngx"
Sep 18 06:26:34.445946 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:34.445877    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-wmngx"]
Sep 18 06:26:34.449399 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:26:34.449376    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf09da463_3ce0_459c_b52c_6e2fea2d1ed8.slice/crio-042b8d4b72d4b597f7199f667e6d9ad77b666b4f21c4115e9454bbe3d479de7c WatchSource:0}: Error finding container 042b8d4b72d4b597f7199f667e6d9ad77b666b4f21c4115e9454bbe3d479de7c: Status 404 returned error can't find the container with id 042b8d4b72d4b597f7199f667e6d9ad77b666b4f21c4115e9454bbe3d479de7c
Sep 18 06:26:34.485065 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:34.485034    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-wmngx" event={"ID":"f09da463-3ce0-459c-b52c-6e2fea2d1ed8","Type":"ContainerStarted","Data":"042b8d4b72d4b597f7199f667e6d9ad77b666b4f21c4115e9454bbe3d479de7c"}
Sep 18 06:26:34.747188 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:34.747109    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:34.921716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:34.921691    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nrnz8_ead9d685-da05-4132-b36c-4887546472f2/dns-node-resolver/0.log"
Sep 18 06:26:35.522233 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:35.522194    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6lmr2_612baaf8-db1e-44ca-993e-836959f23b2d/node-ca/0.log"
Sep 18 06:26:35.648140 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:35.648105    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f412820-0d80-43dc-94bd-fd4b5a773fde-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:35.648332 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:35.648259    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 06:26:35.648332 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:35.648273    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f412820-0d80-43dc-94bd-fd4b5a773fde-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:35.648332 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:35.648332    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9f412820-0d80-43dc-94bd-fd4b5a773fde-kube-rbac-proxy-cm podName:9f412820-0d80-43dc-94bd-fd4b5a773fde nodeName:}" failed. No retries permitted until 2026-09-18 06:26:39.648314765 +0000 UTC m=+265.329842503 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9f412820-0d80-43dc-94bd-fd4b5a773fde-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rqpls" (UID: "9f412820-0d80-43dc-94bd-fd4b5a773fde") : configmap "kube-rbac-proxy" not found
Sep 18 06:26:35.648524 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:35.648405    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 06:26:35.648524 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:35.648461    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f412820-0d80-43dc-94bd-fd4b5a773fde-insights-runtime-extractor-tls podName:9f412820-0d80-43dc-94bd-fd4b5a773fde nodeName:}" failed. No retries permitted until 2026-09-18 06:26:39.648447834 +0000 UTC m=+265.329975557 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9f412820-0d80-43dc-94bd-fd4b5a773fde-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rqpls" (UID: "9f412820-0d80-43dc-94bd-fd4b5a773fde") : secret "insights-runtime-extractor-tls" not found
Sep 18 06:26:36.722253 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:36.722162    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wz2zs_d1f10a14-3b4a-4232-a2b0-5328c2384cb8/serve-healthcheck-canary/0.log"
Sep 18 06:26:37.078210 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:37.078129    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:37.495809 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:37.495767    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-wmngx" event={"ID":"f09da463-3ce0-459c-b52c-6e2fea2d1ed8","Type":"ContainerStarted","Data":"d65f218e36c3fa6e7084225a61c561e572d9e7152e4d80e815fed35c3b7d11ca"}
Sep 18 06:26:37.511696 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:37.511651    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-55f6fc7b86-wmngx" podStartSLOduration=2.507041757 podStartE2EDuration="4.511637233s" totalImagesPullingTime="2.004595476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:26:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:26:37.510190508 +0000 UTC m=+263.191718253" watchObservedRunningTime="2026-09-18 06:26:37.511637233 +0000 UTC m=+263.193164977"
Sep 18 06:26:38.194636 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:38.194594    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:38.295651 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:38.295613    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:38.395574 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:38.395544    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:38.445450 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:38.445375    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:38.609097 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:38.609069    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:38.897682 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:38.897649    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:39.680415 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:39.680377    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f412820-0d80-43dc-94bd-fd4b5a773fde-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:39.680841 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:39.680541    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 06:26:39.680841 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:39.680614    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9f412820-0d80-43dc-94bd-fd4b5a773fde-kube-rbac-proxy-cm podName:9f412820-0d80-43dc-94bd-fd4b5a773fde nodeName:}" failed. No retries permitted until 2026-09-18 06:26:47.680601086 +0000 UTC m=+273.362128812 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9f412820-0d80-43dc-94bd-fd4b5a773fde-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rqpls" (UID: "9f412820-0d80-43dc-94bd-fd4b5a773fde") : configmap "kube-rbac-proxy" not found
Sep 18 06:26:39.680841 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:39.680654    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f412820-0d80-43dc-94bd-fd4b5a773fde-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:39.680841 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:39.680743    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 06:26:39.680841 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:39.680789    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f412820-0d80-43dc-94bd-fd4b5a773fde-insights-runtime-extractor-tls podName:9f412820-0d80-43dc-94bd-fd4b5a773fde nodeName:}" failed. No retries permitted until 2026-09-18 06:26:47.680773769 +0000 UTC m=+273.362301495 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9f412820-0d80-43dc-94bd-fd4b5a773fde-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rqpls" (UID: "9f412820-0d80-43dc-94bd-fd4b5a773fde") : secret "insights-runtime-extractor-tls" not found
Sep 18 06:26:39.696693 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:39.696508    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:40.097101 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:40.097012    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:40.494120 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:40.494085    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:40.894955 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:40.894880    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:41.295841 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:41.295812    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:41.403133 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:41.403102    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:41.497318 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:41.497291    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:41.901993 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:41.901966    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:42.105169 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:42.105141    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:42.287846 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:42.287768    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:42.533621 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:42.533595    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:43.094824 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:43.094799    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:43.295659 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:43.295629    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:43.498002 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:43.497970    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:43.895785 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:43.895706    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:44.499172 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:44.499134    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:44.694751 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:44.694723    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:44.795531 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:44.795461    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:45.304623 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:45.304590    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:45.903467 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:45.903441    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:46.098110 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:46.098083    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:46.296710 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:46.296607    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:46.698201 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:46.698159    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:47.295713 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:47.295679    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:47.706522 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:47.706494    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:47.738867 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:47.738844    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f412820-0d80-43dc-94bd-fd4b5a773fde-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:47.738963 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:47.738889    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f412820-0d80-43dc-94bd-fd4b5a773fde-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:47.739016 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:47.738993    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 06:26:47.739108 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:47.739096    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f412820-0d80-43dc-94bd-fd4b5a773fde-insights-runtime-extractor-tls podName:9f412820-0d80-43dc-94bd-fd4b5a773fde nodeName:}" failed. No retries permitted until 2026-09-18 06:27:03.739081532 +0000 UTC m=+289.420609266 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9f412820-0d80-43dc-94bd-fd4b5a773fde-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rqpls" (UID: "9f412820-0d80-43dc-94bd-fd4b5a773fde") : secret "insights-runtime-extractor-tls" not found
Sep 18 06:26:47.739388 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:47.739365    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9f412820-0d80-43dc-94bd-fd4b5a773fde-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:26:48.107133 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:48.107052    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:48.498162 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:48.498134    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:48.701131 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:48.701100    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:48.898084 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:48.898057    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:51.427178 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:51.427147    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:51.478752 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:51.478723    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:56.487836 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.487807    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-db56994c6-v26pk"]
Sep 18 06:26:56.493186 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.493167    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.495727 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.495700    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 18 06:26:56.495891 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.495701    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 18 06:26:56.497384 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.496866    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 18 06:26:56.497384 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.497172    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-d7llw\""
Sep 18 06:26:56.501602 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.501585    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 18 06:26:56.501882 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.501852    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-db56994c6-v26pk"]
Sep 18 06:26:56.555332 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.555305    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:56.599639 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.599609    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-ca-trust-extracted\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.599779 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.599649    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-registry-certificates\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.599779 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.599686    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-trusted-ca\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.599779 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.599722    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-image-registry-private-configuration\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.599779 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.599771    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-bound-sa-token\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.599939 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.599811    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-installation-pull-secrets\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.599939 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.599854    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbfw5\" (UniqueName: \"kubernetes.io/projected/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-kube-api-access-mbfw5\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.599939 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.599901    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-registry-tls\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.652691 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.652665    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:26:56.700369 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.700347    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-image-registry-private-configuration\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.700467 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.700380    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-bound-sa-token\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.700529 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.700509    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-installation-pull-secrets\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.700562 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.700545    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbfw5\" (UniqueName: \"kubernetes.io/projected/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-kube-api-access-mbfw5\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.700603 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.700576    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-registry-tls\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.700637 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.700627    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-ca-trust-extracted\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.700676 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.700663    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-registry-certificates\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.700713 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.700698    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-trusted-ca\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.701122 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.701095    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-ca-trust-extracted\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.701520 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.701501    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-registry-certificates\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.701845 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.701823    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-trusted-ca\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.702817 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.702795    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-image-registry-private-configuration\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.702989 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.702973    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-registry-tls\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.703087 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.703068    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-installation-pull-secrets\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.708054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.708035    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbfw5\" (UniqueName: \"kubernetes.io/projected/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-kube-api-access-mbfw5\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.708165 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.708144    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2-bound-sa-token\") pod \"image-registry-db56994c6-v26pk\" (UID: \"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2\") " pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:56.805741 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:56.805686    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:57.072557 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:57.072323    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-db56994c6-v26pk"]
Sep 18 06:26:57.074819 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:26:57.074788    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b8ab2d6_1229_40ec_ae1e_4e2d2f7f66b2.slice/crio-ce7f2c946b87f8a6e7aeed00c68c976341e5d57ecfea12fd0868b35c563508b2 WatchSource:0}: Error finding container ce7f2c946b87f8a6e7aeed00c68c976341e5d57ecfea12fd0868b35c563508b2: Status 404 returned error can't find the container with id ce7f2c946b87f8a6e7aeed00c68c976341e5d57ecfea12fd0868b35c563508b2
Sep 18 06:26:57.215050 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:26:57.215009    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-jdjzn" podUID="4a5817a8-34ca-4d89-8543-76631f2d2d62"
Sep 18 06:26:57.550045 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:57.550007    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db56994c6-v26pk" event={"ID":"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2","Type":"ContainerStarted","Data":"4a4fd9a136cdaad767f4b9792efb292da58fcc49e2dae5aa0a9c579babfcbf7f"}
Sep 18 06:26:57.550443 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:57.550061    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-db56994c6-v26pk" event={"ID":"3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2","Type":"ContainerStarted","Data":"ce7f2c946b87f8a6e7aeed00c68c976341e5d57ecfea12fd0868b35c563508b2"}
Sep 18 06:26:57.550443 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:57.550025    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:26:57.550443 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:57.550117    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:26:57.567886 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:26:57.567845    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-db56994c6-v26pk" podStartSLOduration=1.567810436 podStartE2EDuration="1.567810436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:26:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:26:57.566937717 +0000 UTC m=+283.248465460" watchObservedRunningTime="2026-09-18 06:26:57.567810436 +0000 UTC m=+283.249338218"
Sep 18 06:27:00.426644 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:00.426599    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls\") pod \"dns-default-jdjzn\" (UID: \"4a5817a8-34ca-4d89-8543-76631f2d2d62\") " pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:27:00.428802 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:00.428781    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4a5817a8-34ca-4d89-8543-76631f2d2d62-metrics-tls\") pod \"dns-default-jdjzn\" (UID: \"4a5817a8-34ca-4d89-8543-76631f2d2d62\") " pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:27:00.553651 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:00.553625    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-sn4gt\""
Sep 18 06:27:00.562095 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:00.562075    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:27:00.828879 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:00.828811    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jdjzn"]
Sep 18 06:27:00.831192 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:27:00.831157    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a5817a8_34ca_4d89_8543_76631f2d2d62.slice/crio-103bfd5ae186617db31a6d9adc3c36b7e8cb837407b7fc2538af14971ada8ed9 WatchSource:0}: Error finding container 103bfd5ae186617db31a6d9adc3c36b7e8cb837407b7fc2538af14971ada8ed9: Status 404 returned error can't find the container with id 103bfd5ae186617db31a6d9adc3c36b7e8cb837407b7fc2538af14971ada8ed9
Sep 18 06:27:01.356019 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:01.355978    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:01.455113 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:01.455077    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:01.518032 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:01.517994    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:01.563132 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:01.563092    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jdjzn" event={"ID":"4a5817a8-34ca-4d89-8543-76631f2d2d62","Type":"ContainerStarted","Data":"103bfd5ae186617db31a6d9adc3c36b7e8cb837407b7fc2538af14971ada8ed9"}
Sep 18 06:27:01.703378 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:01.703351    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:03.570741 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:03.570707    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jdjzn" event={"ID":"4a5817a8-34ca-4d89-8543-76631f2d2d62","Type":"ContainerStarted","Data":"c24558729398871b458c5a8aeb3bb3815f52fccfd451ea609df703f5c2948221"}
Sep 18 06:27:03.571179 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:03.570764    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jdjzn" event={"ID":"4a5817a8-34ca-4d89-8543-76631f2d2d62","Type":"ContainerStarted","Data":"2e0f6566c54fc77d8a1e01891b7a1d84983d76356b724d45110db5d35e5906ed"}
Sep 18 06:27:03.571179 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:03.570842    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:27:03.586362 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:03.586319    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jdjzn" podStartSLOduration=251.853226013 podStartE2EDuration="4m13.586304562s" totalImagesPullingTime="1.733078549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:27:03.585388994 +0000 UTC m=+289.266916752" watchObservedRunningTime="2026-09-18 06:27:03.586304562 +0000 UTC m=+289.267832306"
Sep 18 06:27:03.754855 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:03.754820    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f412820-0d80-43dc-94bd-fd4b5a773fde-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:27:03.757119 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:03.757094    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9f412820-0d80-43dc-94bd-fd4b5a773fde-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rqpls\" (UID: \"9f412820-0d80-43dc-94bd-fd4b5a773fde\") " pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:27:03.765096 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:03.765075    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rqpls"
Sep 18 06:27:04.031912 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:04.031812    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rqpls"]
Sep 18 06:27:04.034780 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:27:04.034756    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f412820_0d80_43dc_94bd_fd4b5a773fde.slice/crio-a0aa98d005db88bf365d565445474eadee4c1c9e7665f44a81e9a70698a4b9ab WatchSource:0}: Error finding container a0aa98d005db88bf365d565445474eadee4c1c9e7665f44a81e9a70698a4b9ab: Status 404 returned error can't find the container with id a0aa98d005db88bf365d565445474eadee4c1c9e7665f44a81e9a70698a4b9ab
Sep 18 06:27:04.527262 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:04.527207    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:04.579798 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:04.579766    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rqpls" event={"ID":"9f412820-0d80-43dc-94bd-fd4b5a773fde","Type":"ContainerStarted","Data":"1c0b5790bcb9fa9e5d3f715514ebcfba0f95e47c16c5c43aaf9d5984b2a083c6"}
Sep 18 06:27:04.580174 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:04.579843    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rqpls" event={"ID":"9f412820-0d80-43dc-94bd-fd4b5a773fde","Type":"ContainerStarted","Data":"a0aa98d005db88bf365d565445474eadee4c1c9e7665f44a81e9a70698a4b9ab"}
Sep 18 06:27:05.583962 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:05.583929    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rqpls" event={"ID":"9f412820-0d80-43dc-94bd-fd4b5a773fde","Type":"ContainerStarted","Data":"bb48a78d0f73297748300b08f5acf1eb4fbd7ba149d8ca1a8a2f8bd22863357e"}
Sep 18 06:27:05.829482 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:05.829450    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:06.588604 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:06.588519    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rqpls" event={"ID":"9f412820-0d80-43dc-94bd-fd4b5a773fde","Type":"ContainerStarted","Data":"2df5bfbd8159c91b3e3658b39954740ed6982882d842bdb315694709985df39a"}
Sep 18 06:27:06.606767 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:06.606719    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rqpls" podStartSLOduration=33.543985944 podStartE2EDuration="35.606705058s" totalImagesPullingTime="2.062719114s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 06:26:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:27:06.605743972 +0000 UTC m=+292.287271716" watchObservedRunningTime="2026-09-18 06:27:06.606705058 +0000 UTC m=+292.288232830"
Sep 18 06:27:07.108743 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:07.108705    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:07.209892 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:07.209851    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:07.312171 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:07.312137    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:07.410616 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:07.410584    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:07.465050 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:07.465026    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:08.038247 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:08.038205    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:08.084985 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:08.084957    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:09.655610 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:09.655579    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:09.892699 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:09.892657    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:09.944060 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:09.943986    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-x4vkx"]
Sep 18 06:27:09.947590 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:09.947568    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:09.950017 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:09.949994    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 18 06:27:09.950510 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:09.950394    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 18 06:27:09.950510 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:09.950474    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 18 06:27:09.951380 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:09.951157    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 18 06:27:09.951380 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:09.951167    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cmnll\""
Sep 18 06:27:09.951380 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:09.951202    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 18 06:27:09.951380 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:09.951367    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 18 06:27:09.975988 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:09.975966    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:09.998996 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:09.998975    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/56815ee5-968b-4af9-95ad-3cf971e1e720-node-exporter-textfile\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:09.999098 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:09.999015    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/56815ee5-968b-4af9-95ad-3cf971e1e720-node-exporter-tls\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:09.999098 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:09.999041    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/56815ee5-968b-4af9-95ad-3cf971e1e720-sys\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:09.999098 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:09.999069    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/56815ee5-968b-4af9-95ad-3cf971e1e720-node-exporter-wtmp\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:09.999242 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:09.999098    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/56815ee5-968b-4af9-95ad-3cf971e1e720-node-exporter-accelerators-collector-config\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:09.999242 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:09.999133    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56815ee5-968b-4af9-95ad-3cf971e1e720-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:09.999242 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:09.999165    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/56815ee5-968b-4af9-95ad-3cf971e1e720-root\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:09.999242 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:09.999210    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b792s\" (UniqueName: \"kubernetes.io/projected/56815ee5-968b-4af9-95ad-3cf971e1e720-kube-api-access-b792s\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:09.999395 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:09.999248    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56815ee5-968b-4af9-95ad-3cf971e1e720-metrics-client-ca\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:10.031189 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.031165    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:10.100575 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.100531    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/56815ee5-968b-4af9-95ad-3cf971e1e720-node-exporter-wtmp\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:10.100726 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.100583    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/56815ee5-968b-4af9-95ad-3cf971e1e720-node-exporter-accelerators-collector-config\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:10.100726 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.100622    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56815ee5-968b-4af9-95ad-3cf971e1e720-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:10.100726 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.100660    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/56815ee5-968b-4af9-95ad-3cf971e1e720-root\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:10.100726 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.100708    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b792s\" (UniqueName: \"kubernetes.io/projected/56815ee5-968b-4af9-95ad-3cf971e1e720-kube-api-access-b792s\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:10.100856 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.100734    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56815ee5-968b-4af9-95ad-3cf971e1e720-metrics-client-ca\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:10.100856 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.100737    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/56815ee5-968b-4af9-95ad-3cf971e1e720-node-exporter-wtmp\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:10.100856 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.100770    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/56815ee5-968b-4af9-95ad-3cf971e1e720-node-exporter-textfile\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:10.100856 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.100803    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/56815ee5-968b-4af9-95ad-3cf971e1e720-root\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:10.100856 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.100805    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/56815ee5-968b-4af9-95ad-3cf971e1e720-node-exporter-tls\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:10.100856 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.100846    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/56815ee5-968b-4af9-95ad-3cf971e1e720-sys\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:10.101028 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:27:10.100898    2568 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 18 06:27:10.101028 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.100911    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/56815ee5-968b-4af9-95ad-3cf971e1e720-sys\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:10.101028 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:27:10.100956    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56815ee5-968b-4af9-95ad-3cf971e1e720-node-exporter-tls podName:56815ee5-968b-4af9-95ad-3cf971e1e720 nodeName:}" failed. No retries permitted until 2026-09-18 06:27:10.600936665 +0000 UTC m=+296.282464400 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/56815ee5-968b-4af9-95ad-3cf971e1e720-node-exporter-tls") pod "node-exporter-x4vkx" (UID: "56815ee5-968b-4af9-95ad-3cf971e1e720") : secret "node-exporter-tls" not found
Sep 18 06:27:10.101350 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.101323    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/56815ee5-968b-4af9-95ad-3cf971e1e720-node-exporter-accelerators-collector-config\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:10.101508 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.101486    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/56815ee5-968b-4af9-95ad-3cf971e1e720-node-exporter-textfile\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:10.101689 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.101669    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56815ee5-968b-4af9-95ad-3cf971e1e720-metrics-client-ca\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:10.103584 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.103564    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56815ee5-968b-4af9-95ad-3cf971e1e720-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:10.112115 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.112089    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b792s\" (UniqueName: \"kubernetes.io/projected/56815ee5-968b-4af9-95ad-3cf971e1e720-kube-api-access-b792s\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:10.151011 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.150968    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:10.250567 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.250496    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:10.348617 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.348569    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:10.604153 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.604079    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/56815ee5-968b-4af9-95ad-3cf971e1e720-node-exporter-tls\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:10.606316 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.606287    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/56815ee5-968b-4af9-95ad-3cf971e1e720-node-exporter-tls\") pod \"node-exporter-x4vkx\" (UID: \"56815ee5-968b-4af9-95ad-3cf971e1e720\") " pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:10.817480 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.817438    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:10.858147 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.858076    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-x4vkx"
Sep 18 06:27:10.868716 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:27:10.868685    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56815ee5_968b_4af9_95ad_3cf971e1e720.slice/crio-0837cf77480faebc4f9ea6e6d703d8785c25cd2ca891d901ebf8f9b7305d6059 WatchSource:0}: Error finding container 0837cf77480faebc4f9ea6e6d703d8785c25cd2ca891d901ebf8f9b7305d6059: Status 404 returned error can't find the container with id 0837cf77480faebc4f9ea6e6d703d8785c25cd2ca891d901ebf8f9b7305d6059
Sep 18 06:27:10.917482 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.917454    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:10.945946 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.945927    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 06:27:10.950304 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.950290    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:10.952672 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.952647    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 18 06:27:10.952768 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.952706    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 18 06:27:10.952843 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.952791    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 18 06:27:10.953083 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.953054    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 18 06:27:10.953083 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.953065    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qr5s8\""
Sep 18 06:27:10.953264 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.953063    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 18 06:27:10.953264 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.953133    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 18 06:27:10.953378 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.953336    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 18 06:27:10.953800 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.953784    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 18 06:27:10.953864 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.953838    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 18 06:27:10.961747 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.961723    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 06:27:10.993124 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:10.993102    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:11.006365 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.006343    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.006457 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.006375    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-web-config\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.006457 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.006396    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j5b8\" (UniqueName: \"kubernetes.io/projected/3fc7afc5-4b83-45aa-9442-a75c500c32c5-kube-api-access-4j5b8\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.006457 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.006424    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3fc7afc5-4b83-45aa-9442-a75c500c32c5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.006457 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.006439    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-config-volume\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.006619 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.006496    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.006619 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.006541    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3fc7afc5-4b83-45aa-9442-a75c500c32c5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.006619 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.006568    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fc7afc5-4b83-45aa-9442-a75c500c32c5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.006619 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.006604    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3fc7afc5-4b83-45aa-9442-a75c500c32c5-config-out\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.006750 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.006633    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fc7afc5-4b83-45aa-9442-a75c500c32c5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.006750 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.006702    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.006750 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.006726    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.006902 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.006750    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.107860 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.107832    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.107860 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.107861    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-web-config\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.108109 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.107884    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j5b8\" (UniqueName: \"kubernetes.io/projected/3fc7afc5-4b83-45aa-9442-a75c500c32c5-kube-api-access-4j5b8\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.108109 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.107913    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3fc7afc5-4b83-45aa-9442-a75c500c32c5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.108109 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.107936    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-config-volume\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.108109 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.107963    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.108109 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.107984    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3fc7afc5-4b83-45aa-9442-a75c500c32c5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.108109 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.108005    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fc7afc5-4b83-45aa-9442-a75c500c32c5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.108109 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.108032    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3fc7afc5-4b83-45aa-9442-a75c500c32c5-config-out\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.108109 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.108064    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fc7afc5-4b83-45aa-9442-a75c500c32c5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.108556 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.108125    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.108556 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.108148    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.108556 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.108172    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.108556 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.108370    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3fc7afc5-4b83-45aa-9442-a75c500c32c5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.108556 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:27:11.108507    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3fc7afc5-4b83-45aa-9442-a75c500c32c5-alertmanager-trusted-ca-bundle podName:3fc7afc5-4b83-45aa-9442-a75c500c32c5 nodeName:}" failed. No retries permitted until 2026-09-18 06:27:11.608487191 +0000 UTC m=+297.290014926 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/3fc7afc5-4b83-45aa-9442-a75c500c32c5-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "3fc7afc5-4b83-45aa-9442-a75c500c32c5") : configmap references non-existent config key: ca-bundle.crt
Sep 18 06:27:11.109307 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.109237    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fc7afc5-4b83-45aa-9442-a75c500c32c5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.111282 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.111233    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3fc7afc5-4b83-45aa-9442-a75c500c32c5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.111405 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.111351    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.111405 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.111363    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.111531 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.111454    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-web-config\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.111588 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.111529    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3fc7afc5-4b83-45aa-9442-a75c500c32c5-config-out\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.111837 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.111817    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.111913 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.111834    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.111972 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.111926    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.112691 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.112669    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-config-volume\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.115438 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.115418    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j5b8\" (UniqueName: \"kubernetes.io/projected/3fc7afc5-4b83-45aa-9442-a75c500c32c5-kube-api-access-4j5b8\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.118724 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.118695    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:11.609116 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.609079    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x4vkx" event={"ID":"56815ee5-968b-4af9-95ad-3cf971e1e720","Type":"ContainerStarted","Data":"0837cf77480faebc4f9ea6e6d703d8785c25cd2ca891d901ebf8f9b7305d6059"}
Sep 18 06:27:11.612994 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.612969    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fc7afc5-4b83-45aa-9442-a75c500c32c5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.613803 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.613773    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fc7afc5-4b83-45aa-9442-a75c500c32c5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:11.860645 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:11.860574    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:27:12.159472 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:12.159442    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 06:27:12.160084 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:27:12.160048    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fc7afc5_4b83_45aa_9442_a75c500c32c5.slice/crio-d310b955b2aeacbad4d1da6637b8e7eee3ff89c327e664423f824da51f96541c WatchSource:0}: Error finding container d310b955b2aeacbad4d1da6637b8e7eee3ff89c327e664423f824da51f96541c: Status 404 returned error can't find the container with id d310b955b2aeacbad4d1da6637b8e7eee3ff89c327e664423f824da51f96541c
Sep 18 06:27:12.537006 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:27:12.536970    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-k6z9c" podUID="bcf277d6-51cd-4bc8-ba3f-e6fcf272d363"
Sep 18 06:27:12.615029 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:12.614987    2568 generic.go:356] "Generic (PLEG): container finished" podID="56815ee5-968b-4af9-95ad-3cf971e1e720" containerID="b83678fc890d90328b88af84c88d98a164f657f6e766d6f51c5c0d60c75db52d" exitCode=0
Sep 18 06:27:12.615198 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:12.615085    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x4vkx" event={"ID":"56815ee5-968b-4af9-95ad-3cf971e1e720","Type":"ContainerDied","Data":"b83678fc890d90328b88af84c88d98a164f657f6e766d6f51c5c0d60c75db52d"}
Sep 18 06:27:12.616333 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:12.616299    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc7afc5-4b83-45aa-9442-a75c500c32c5","Type":"ContainerStarted","Data":"d310b955b2aeacbad4d1da6637b8e7eee3ff89c327e664423f824da51f96541c"}
Sep 18 06:27:12.819241 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:12.819148    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:12.896980 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:12.896948    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:13.000401 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:13.000364    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:13.398637 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:13.398611    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:13.581915 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:13.581887    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jdjzn"
Sep 18 06:27:13.622018 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:13.621988    2568 generic.go:356] "Generic (PLEG): container finished" podID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerID="9cdcf778672dd29eb9d2f791e6de32a05d43bf3658e8baffa564bfe6e962732e" exitCode=0
Sep 18 06:27:13.622177 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:13.622063    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc7afc5-4b83-45aa-9442-a75c500c32c5","Type":"ContainerDied","Data":"9cdcf778672dd29eb9d2f791e6de32a05d43bf3658e8baffa564bfe6e962732e"}
Sep 18 06:27:13.624528 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:13.624509    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x4vkx" event={"ID":"56815ee5-968b-4af9-95ad-3cf971e1e720","Type":"ContainerStarted","Data":"7e7b17c83926068c35cf683ede97274de04cb03b99766b99c8104448728fc1af"}
Sep 18 06:27:13.624631 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:13.624549    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x4vkx" event={"ID":"56815ee5-968b-4af9-95ad-3cf971e1e720","Type":"ContainerStarted","Data":"1a4b1ef06bd6592fab20093eb060000ac0c59549f1da167854a9e3cc370eddd2"}
Sep 18 06:27:13.668071 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:13.667985    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-x4vkx" podStartSLOduration=3.591247001 podStartE2EDuration="4.667972196s" totalImagesPullingTime="1.017910711s" totalInitContainerRuntime="58.814484ms" isStatefulPod=true podCreationTimestamp="2026-09-18 06:27:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:27:13.666827892 +0000 UTC m=+299.348355636" watchObservedRunningTime="2026-09-18 06:27:13.667972196 +0000 UTC m=+299.349499981"
Sep 18 06:27:14.667062 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:14.667032    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:15.083603 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:15.083559    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:15.635647 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:15.635571    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc7afc5-4b83-45aa-9442-a75c500c32c5","Type":"ContainerStarted","Data":"30a51a79cd83f038afc473e0ed73ecf29b627e19d5779e1146ab41bdd47f171d"}
Sep 18 06:27:15.635793 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:15.635683    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc7afc5-4b83-45aa-9442-a75c500c32c5","Type":"ContainerStarted","Data":"2a9b265edcd5951a41ef029813295e2dba4e91982ff090a55f0fa4c1d4a2014b"}
Sep 18 06:27:15.635793 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:15.635723    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc7afc5-4b83-45aa-9442-a75c500c32c5","Type":"ContainerStarted","Data":"3af7a4d343f51b4a24eceb49bc88a6a6086c0a7fa696d171ff71bf3286c761a1"}
Sep 18 06:27:15.635793 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:15.635754    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc7afc5-4b83-45aa-9442-a75c500c32c5","Type":"ContainerStarted","Data":"a1272b1b065518f705e9bfa4ada9969df25e3b2e99a0a8d792279542ef74cea5"}
Sep 18 06:27:15.635793 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:15.635789    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc7afc5-4b83-45aa-9442-a75c500c32c5","Type":"ContainerStarted","Data":"c9182d3c862f202444af58eb0a9e78157b673da5e841349b726cf93493689173"}
Sep 18 06:27:16.059088 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:16.059056    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:16.405535 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:16.405513    2568 kubelet.go:1648] "Image garbage collection succeeded"
Sep 18 06:27:16.642586 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:16.642437    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc7afc5-4b83-45aa-9442-a75c500c32c5","Type":"ContainerStarted","Data":"d274c1f8fecd3e655bfdebbeb55be3a30259abccfecd88f799245fe69cef1b32"}
Sep 18 06:27:16.674886 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:16.674807    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.665545795 podStartE2EDuration="6.674792407s" totalImagesPullingTime="2.998057038s" totalInitContainerRuntime="11.189574ms" isStatefulPod=false podCreationTimestamp="2026-09-18 06:27:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:27:16.673903392 +0000 UTC m=+302.355431138" watchObservedRunningTime="2026-09-18 06:27:16.674792407 +0000 UTC m=+302.356320212"
Sep 18 06:27:17.151730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:17.151700    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:17.262245 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:17.262206    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:18.556889 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:18.556864    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-db56994c6-v26pk"
Sep 18 06:27:18.670154 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:18.670124    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:18.770392 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:18.770366    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:21.430186 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:21.430150    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:21.482851 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:21.482821    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:22.689088 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:22.689057    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:22.735333 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:22.735306    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:23.537272 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:23.537239    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:27:23.806129 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:23.806057    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6594f4c75-8l8dt"]
Sep 18 06:27:23.809208 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:23.809192    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-8l8dt"
Sep 18 06:27:23.811435 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:23.811417    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 18 06:27:23.811568 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:23.811551    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-jvzbd\""
Sep 18 06:27:23.811568 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:23.811561    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 18 06:27:23.818130 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:23.818111    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-8l8dt"]
Sep 18 06:27:23.821147 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:23.821125    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:23.912549 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:23.912523    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67ln8\" (UniqueName: \"kubernetes.io/projected/05b3941b-0717-4241-9cf7-a45764dcb8db-kube-api-access-67ln8\") pod \"downloads-6594f4c75-8l8dt\" (UID: \"05b3941b-0717-4241-9cf7-a45764dcb8db\") " pod="openshift-console/downloads-6594f4c75-8l8dt"
Sep 18 06:27:23.917928 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:23.917906    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:23.964072 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:23.964048    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:24.013319 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:24.013296    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67ln8\" (UniqueName: \"kubernetes.io/projected/05b3941b-0717-4241-9cf7-a45764dcb8db-kube-api-access-67ln8\") pod \"downloads-6594f4c75-8l8dt\" (UID: \"05b3941b-0717-4241-9cf7-a45764dcb8db\") " pod="openshift-console/downloads-6594f4c75-8l8dt"
Sep 18 06:27:24.020462 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:24.020429    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67ln8\" (UniqueName: \"kubernetes.io/projected/05b3941b-0717-4241-9cf7-a45764dcb8db-kube-api-access-67ln8\") pod \"downloads-6594f4c75-8l8dt\" (UID: \"05b3941b-0717-4241-9cf7-a45764dcb8db\") " pod="openshift-console/downloads-6594f4c75-8l8dt"
Sep 18 06:27:24.117781 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:24.117693    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-8l8dt"
Sep 18 06:27:24.118519 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:24.118496    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:24.384907 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:24.384884    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-8l8dt"]
Sep 18 06:27:24.387407 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:27:24.387381    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05b3941b_0717_4241_9cf7_a45764dcb8db.slice/crio-9fb3f4232c13c6853b8863dea55f23dd84d02242352bc5db800baf1147fc398c WatchSource:0}: Error finding container 9fb3f4232c13c6853b8863dea55f23dd84d02242352bc5db800baf1147fc398c: Status 404 returned error can't find the container with id 9fb3f4232c13c6853b8863dea55f23dd84d02242352bc5db800baf1147fc398c
Sep 18 06:27:24.389523 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:24.389506    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 06:27:24.490123 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:24.490094    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:24.587839 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:24.587790    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:24.668740 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:24.668707    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-8l8dt" event={"ID":"05b3941b-0717-4241-9cf7-a45764dcb8db","Type":"ContainerStarted","Data":"9fb3f4232c13c6853b8863dea55f23dd84d02242352bc5db800baf1147fc398c"}
Sep 18 06:27:24.889824 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:24.889790    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:24.987427 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:24.987350    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:25.697439 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:25.697408    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:25.898601 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:25.898564    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:26.006455 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:26.006350    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:26.715160 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:26.715119    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:26.768232 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:26.768187    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:31.104263 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:31.104227    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:31.160522 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:31.160489    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:31.703924 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:31.703890    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:31.805985 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:31.805953    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:33.924799 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:33.924759    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:33.970247 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:33.970199    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:36.429763 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:36.429726    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:36.498518 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:36.498485    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:36.717284 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:36.717151    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:36.978699 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:36.978584    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wz2zs_d1f10a14-3b4a-4232-a2b0-5328c2384cb8/serve-healthcheck-canary/0.log"
Sep 18 06:27:40.730614 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:40.730565    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-8l8dt" event={"ID":"05b3941b-0717-4241-9cf7-a45764dcb8db","Type":"ContainerStarted","Data":"910fcf776e6169df92789869faaf85ff668c9810b369d77d303a6c0dc72b75a3"}
Sep 18 06:27:40.731115 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:40.730819    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6594f4c75-8l8dt"
Sep 18 06:27:40.750557 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:40.750532    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6594f4c75-8l8dt"
Sep 18 06:27:40.767031 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:40.766978    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6594f4c75-8l8dt" podStartSLOduration=1.938301898 podStartE2EDuration="17.766961152s" totalImagesPullingTime="15.828659254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:27:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:27:40.74978934 +0000 UTC m=+326.431317085" watchObservedRunningTime="2026-09-18 06:27:40.766961152 +0000 UTC m=+326.448488896"
Sep 18 06:27:40.918750 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:40.918704    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:40.969850 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:40.969813    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:44.242345 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:44.242291    2568 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl" podUID="743c3866-3864-4879-8784-1dbf1a28414f" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 18 06:27:45.918458 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:45.918421    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:45.967308 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:45.967277    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:46.310000 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.309925    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5fbbd7b6d6-n25dt"]
Sep 18 06:27:46.340240 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.340200    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fbbd7b6d6-n25dt"]
Sep 18 06:27:46.340405 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.340328    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.343158 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.343136    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 18 06:27:46.344484 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.344152    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 18 06:27:46.344484 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.344237    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 18 06:27:46.344484 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.344166    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-q62qv\""
Sep 18 06:27:46.344484 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.344279    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 18 06:27:46.344484 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.344341    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 18 06:27:46.354025 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.354002    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 18 06:27:46.507788 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.507756    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-trusted-ca-bundle\") pod \"console-5fbbd7b6d6-n25dt\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") " pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.507965 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.507800    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c8ed825-382d-4cbe-9342-f1ce600bed96-console-serving-cert\") pod \"console-5fbbd7b6d6-n25dt\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") " pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.507965 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.507825    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-oauth-serving-cert\") pod \"console-5fbbd7b6d6-n25dt\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") " pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.507965 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.507913    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c8ed825-382d-4cbe-9342-f1ce600bed96-console-oauth-config\") pod \"console-5fbbd7b6d6-n25dt\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") " pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.508114 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.507965    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-service-ca\") pod \"console-5fbbd7b6d6-n25dt\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") " pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.508114 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.508016    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbcc2\" (UniqueName: \"kubernetes.io/projected/5c8ed825-382d-4cbe-9342-f1ce600bed96-kube-api-access-qbcc2\") pod \"console-5fbbd7b6d6-n25dt\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") " pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.508114 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.508093    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-console-config\") pod \"console-5fbbd7b6d6-n25dt\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") " pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.609328 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.609245    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c8ed825-382d-4cbe-9342-f1ce600bed96-console-oauth-config\") pod \"console-5fbbd7b6d6-n25dt\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") " pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.609328 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.609282    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-service-ca\") pod \"console-5fbbd7b6d6-n25dt\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") " pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.609328 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.609320    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbcc2\" (UniqueName: \"kubernetes.io/projected/5c8ed825-382d-4cbe-9342-f1ce600bed96-kube-api-access-qbcc2\") pod \"console-5fbbd7b6d6-n25dt\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") " pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.609604 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.609352    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-console-config\") pod \"console-5fbbd7b6d6-n25dt\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") " pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.609604 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.609418    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-trusted-ca-bundle\") pod \"console-5fbbd7b6d6-n25dt\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") " pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.609604 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.609452    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c8ed825-382d-4cbe-9342-f1ce600bed96-console-serving-cert\") pod \"console-5fbbd7b6d6-n25dt\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") " pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.609604 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.609474    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-oauth-serving-cert\") pod \"console-5fbbd7b6d6-n25dt\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") " pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.610256 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.610206    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-service-ca\") pod \"console-5fbbd7b6d6-n25dt\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") " pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.610410 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.610243    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-oauth-serving-cert\") pod \"console-5fbbd7b6d6-n25dt\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") " pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.610504 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.610435    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-console-config\") pod \"console-5fbbd7b6d6-n25dt\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") " pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.610783 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.610762    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-trusted-ca-bundle\") pod \"console-5fbbd7b6d6-n25dt\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") " pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.612156 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.612134    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c8ed825-382d-4cbe-9342-f1ce600bed96-console-oauth-config\") pod \"console-5fbbd7b6d6-n25dt\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") " pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.612382 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.612356    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c8ed825-382d-4cbe-9342-f1ce600bed96-console-serving-cert\") pod \"console-5fbbd7b6d6-n25dt\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") " pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.617207 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.617187    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbcc2\" (UniqueName: \"kubernetes.io/projected/5c8ed825-382d-4cbe-9342-f1ce600bed96-kube-api-access-qbcc2\") pod \"console-5fbbd7b6d6-n25dt\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") " pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.657108 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.657078    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:46.962866 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:46.962743    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5fbbd7b6d6-n25dt"]
Sep 18 06:27:46.965415 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:27:46.965387    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c8ed825_382d_4cbe_9342_f1ce600bed96.slice/crio-8380c286d865ba5c19298d478c89da2d494cff18f9f9825c5a0000169dcb419d WatchSource:0}: Error finding container 8380c286d865ba5c19298d478c89da2d494cff18f9f9825c5a0000169dcb419d: Status 404 returned error can't find the container with id 8380c286d865ba5c19298d478c89da2d494cff18f9f9825c5a0000169dcb419d
Sep 18 06:27:47.762775 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:47.762696    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fbbd7b6d6-n25dt" event={"ID":"5c8ed825-382d-4cbe-9342-f1ce600bed96","Type":"ContainerStarted","Data":"8380c286d865ba5c19298d478c89da2d494cff18f9f9825c5a0000169dcb419d"}
Sep 18 06:27:50.331070 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:50.331031    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:50.384685 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:50.384657    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:51.428461 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:51.428420    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:51.481360 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:51.481325    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:52.114582 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:52.114549    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:52.165263 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:52.165232    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:52.793873 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:52.793836    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fbbd7b6d6-n25dt" event={"ID":"5c8ed825-382d-4cbe-9342-f1ce600bed96","Type":"ContainerStarted","Data":"9ea77edc37d3027382b3c65dc8ee44338f2b8122b3225ea0a1547d34b2778ab4"}
Sep 18 06:27:52.811038 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:52.810989    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5fbbd7b6d6-n25dt" podStartSLOduration=1.518112321 podStartE2EDuration="6.810972807s" totalImagesPullingTime="5.292860486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:27:52.80895038 +0000 UTC m=+338.490478123" watchObservedRunningTime="2026-09-18 06:27:52.810972807 +0000 UTC m=+338.492500550"
Sep 18 06:27:54.241905 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:54.241865    2568 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl" podUID="743c3866-3864-4879-8784-1dbf1a28414f" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 18 06:27:56.658123 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:56.658090    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:56.658549 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:56.658143    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:56.662704 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:56.662684    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:56.811883 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:56.811853    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:27:58.120856 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:58.120825    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:58.170344 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:58.170309    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:59.121427 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:59.121398    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:27:59.171029 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:27:59.170998    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:02.116678 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:02.116639    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:02.164537 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:02.164509    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:04.242707 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:04.242669    2568 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl" podUID="743c3866-3864-4879-8784-1dbf1a28414f" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 18 06:28:04.243174 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:04.242754    2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl"
Sep 18 06:28:04.243377 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:04.243344    2568 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"bd27031aa555b5509eb4d33346eafc9e420e5cf6e1f1ecfba628656bd973c97d"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Sep 18 06:28:04.243491 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:04.243404    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl" podUID="743c3866-3864-4879-8784-1dbf1a28414f" containerName="service-proxy" containerID="cri-o://bd27031aa555b5509eb4d33346eafc9e420e5cf6e1f1ecfba628656bd973c97d" gracePeriod=30
Sep 18 06:28:04.836325 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:04.836291    2568 generic.go:356] "Generic (PLEG): container finished" podID="743c3866-3864-4879-8784-1dbf1a28414f" containerID="bd27031aa555b5509eb4d33346eafc9e420e5cf6e1f1ecfba628656bd973c97d" exitCode=2
Sep 18 06:28:04.836513 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:04.836334    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl" event={"ID":"743c3866-3864-4879-8784-1dbf1a28414f","Type":"ContainerDied","Data":"bd27031aa555b5509eb4d33346eafc9e420e5cf6e1f1ecfba628656bd973c97d"}
Sep 18 06:28:04.836513 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:04.836376    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5645ddf6bc-kbfvl" event={"ID":"743c3866-3864-4879-8784-1dbf1a28414f","Type":"ContainerStarted","Data":"1dbf5af08840e1c2da7c46cd43ad90fe268da3bf991401c3508c46de98948df0"}
Sep 18 06:28:06.427454 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:06.427422    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:06.478104 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:06.478073    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:10.506472 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:10.506435    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:10.607190 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:10.607162    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:10.653794 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:10.653759    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:17.409377 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:17.409343    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:17.464604 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:17.464572    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:29.064092 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:29.064060    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:29.441598 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:29.441563    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:30.105941 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:30.105908    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 06:28:30.106393 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:30.106237    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="alertmanager" containerID="cri-o://c9182d3c862f202444af58eb0a9e78157b673da5e841349b726cf93493689173" gracePeriod=120
Sep 18 06:28:30.106393 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:30.106329    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="kube-rbac-proxy-metric" containerID="cri-o://30a51a79cd83f038afc473e0ed73ecf29b627e19d5779e1146ab41bdd47f171d" gracePeriod=120
Sep 18 06:28:30.106393 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:30.106373    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="kube-rbac-proxy" containerID="cri-o://2a9b265edcd5951a41ef029813295e2dba4e91982ff090a55f0fa4c1d4a2014b" gracePeriod=120
Sep 18 06:28:30.106552 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:30.106323    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="kube-rbac-proxy-web" containerID="cri-o://3af7a4d343f51b4a24eceb49bc88a6a6086c0a7fa696d171ff71bf3286c761a1" gracePeriod=120
Sep 18 06:28:30.106552 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:30.106338    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="prom-label-proxy" containerID="cri-o://d274c1f8fecd3e655bfdebbeb55be3a30259abccfecd88f799245fe69cef1b32" gracePeriod=120
Sep 18 06:28:30.106552 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:30.106364    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="config-reloader" containerID="cri-o://a1272b1b065518f705e9bfa4ada9969df25e3b2e99a0a8d792279542ef74cea5" gracePeriod=120
Sep 18 06:28:30.480090 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:30.480061    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs\") pod \"network-metrics-daemon-k6z9c\" (UID: \"bcf277d6-51cd-4bc8-ba3f-e6fcf272d363\") " pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:28:30.482289 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:30.482262    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcf277d6-51cd-4bc8-ba3f-e6fcf272d363-metrics-certs\") pod \"network-metrics-daemon-k6z9c\" (UID: \"bcf277d6-51cd-4bc8-ba3f-e6fcf272d363\") " pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:28:30.740706 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:30.740633    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fg7cb\""
Sep 18 06:28:30.749462 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:30.749441    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6z9c"
Sep 18 06:28:30.933498 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:30.933468    2568 generic.go:356] "Generic (PLEG): container finished" podID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerID="d274c1f8fecd3e655bfdebbeb55be3a30259abccfecd88f799245fe69cef1b32" exitCode=0
Sep 18 06:28:30.933498 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:30.933491    2568 generic.go:356] "Generic (PLEG): container finished" podID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerID="2a9b265edcd5951a41ef029813295e2dba4e91982ff090a55f0fa4c1d4a2014b" exitCode=0
Sep 18 06:28:30.933498 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:30.933496    2568 generic.go:356] "Generic (PLEG): container finished" podID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerID="a1272b1b065518f705e9bfa4ada9969df25e3b2e99a0a8d792279542ef74cea5" exitCode=0
Sep 18 06:28:30.933498 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:30.933503    2568 generic.go:356] "Generic (PLEG): container finished" podID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerID="c9182d3c862f202444af58eb0a9e78157b673da5e841349b726cf93493689173" exitCode=0
Sep 18 06:28:30.933750 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:30.933552    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc7afc5-4b83-45aa-9442-a75c500c32c5","Type":"ContainerDied","Data":"d274c1f8fecd3e655bfdebbeb55be3a30259abccfecd88f799245fe69cef1b32"}
Sep 18 06:28:30.933750 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:30.933620    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc7afc5-4b83-45aa-9442-a75c500c32c5","Type":"ContainerDied","Data":"2a9b265edcd5951a41ef029813295e2dba4e91982ff090a55f0fa4c1d4a2014b"}
Sep 18 06:28:30.933750 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:30.933647    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc7afc5-4b83-45aa-9442-a75c500c32c5","Type":"ContainerDied","Data":"a1272b1b065518f705e9bfa4ada9969df25e3b2e99a0a8d792279542ef74cea5"}
Sep 18 06:28:30.933750 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:30.933672    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc7afc5-4b83-45aa-9442-a75c500c32c5","Type":"ContainerDied","Data":"c9182d3c862f202444af58eb0a9e78157b673da5e841349b726cf93493689173"}
Sep 18 06:28:31.015132 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.015059    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k6z9c"]
Sep 18 06:28:31.018531 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:28:31.018501    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcf277d6_51cd_4bc8_ba3f_e6fcf272d363.slice/crio-743139f54c7f6a9b93a0a83d71d0d616010693e4f21d7860baa8ce4c908e78dc WatchSource:0}: Error finding container 743139f54c7f6a9b93a0a83d71d0d616010693e4f21d7860baa8ce4c908e78dc: Status 404 returned error can't find the container with id 743139f54c7f6a9b93a0a83d71d0d616010693e4f21d7860baa8ce4c908e78dc
Sep 18 06:28:31.335010 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.334988    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:31.486778 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.486740    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") "
Sep 18 06:28:31.486954 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.486795    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fc7afc5-4b83-45aa-9442-a75c500c32c5-alertmanager-trusted-ca-bundle\") pod \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") "
Sep 18 06:28:31.486954 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.486826    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3fc7afc5-4b83-45aa-9442-a75c500c32c5-config-out\") pod \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") "
Sep 18 06:28:31.486954 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.486856    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fc7afc5-4b83-45aa-9442-a75c500c32c5-metrics-client-ca\") pod \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") "
Sep 18 06:28:31.486954 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.486884    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3fc7afc5-4b83-45aa-9442-a75c500c32c5-alertmanager-main-db\") pod \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") "
Sep 18 06:28:31.486954 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.486940    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4j5b8\" (UniqueName: \"kubernetes.io/projected/3fc7afc5-4b83-45aa-9442-a75c500c32c5-kube-api-access-4j5b8\") pod \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") "
Sep 18 06:28:31.487236 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.486965    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-cluster-tls-config\") pod \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") "
Sep 18 06:28:31.487236 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.486996    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-kube-rbac-proxy-web\") pod \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") "
Sep 18 06:28:31.487236 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.487029    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-kube-rbac-proxy\") pod \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") "
Sep 18 06:28:31.487236 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.487059    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3fc7afc5-4b83-45aa-9442-a75c500c32c5-tls-assets\") pod \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") "
Sep 18 06:28:31.487236 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.487098    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-web-config\") pod \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") "
Sep 18 06:28:31.487236 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.487124    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-config-volume\") pod \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") "
Sep 18 06:28:31.487236 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.487182    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-main-tls\") pod \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\" (UID: \"3fc7afc5-4b83-45aa-9442-a75c500c32c5\") "
Sep 18 06:28:31.487575 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.487249    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fc7afc5-4b83-45aa-9442-a75c500c32c5-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "3fc7afc5-4b83-45aa-9442-a75c500c32c5" (UID: "3fc7afc5-4b83-45aa-9442-a75c500c32c5"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 06:28:31.487575 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.487404    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fc7afc5-4b83-45aa-9442-a75c500c32c5-metrics-client-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:28:31.488447 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.488419    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc7afc5-4b83-45aa-9442-a75c500c32c5-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "3fc7afc5-4b83-45aa-9442-a75c500c32c5" (UID: "3fc7afc5-4b83-45aa-9442-a75c500c32c5"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:28:31.488850 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.488824    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fc7afc5-4b83-45aa-9442-a75c500c32c5-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "3fc7afc5-4b83-45aa-9442-a75c500c32c5" (UID: "3fc7afc5-4b83-45aa-9442-a75c500c32c5"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 06:28:31.490671 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.490551    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc7afc5-4b83-45aa-9442-a75c500c32c5-config-out" (OuterVolumeSpecName: "config-out") pod "3fc7afc5-4b83-45aa-9442-a75c500c32c5" (UID: "3fc7afc5-4b83-45aa-9442-a75c500c32c5"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:28:31.490671 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.490558    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "3fc7afc5-4b83-45aa-9442-a75c500c32c5" (UID: "3fc7afc5-4b83-45aa-9442-a75c500c32c5"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:28:31.490838 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.490710    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-config-volume" (OuterVolumeSpecName: "config-volume") pod "3fc7afc5-4b83-45aa-9442-a75c500c32c5" (UID: "3fc7afc5-4b83-45aa-9442-a75c500c32c5"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:28:31.491294 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.491257    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "3fc7afc5-4b83-45aa-9442-a75c500c32c5" (UID: "3fc7afc5-4b83-45aa-9442-a75c500c32c5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:28:31.491773 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.491744    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "3fc7afc5-4b83-45aa-9442-a75c500c32c5" (UID: "3fc7afc5-4b83-45aa-9442-a75c500c32c5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:28:31.492114 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.492049    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fc7afc5-4b83-45aa-9442-a75c500c32c5-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "3fc7afc5-4b83-45aa-9442-a75c500c32c5" (UID: "3fc7afc5-4b83-45aa-9442-a75c500c32c5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:28:31.492193 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.492151    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fc7afc5-4b83-45aa-9442-a75c500c32c5-kube-api-access-4j5b8" (OuterVolumeSpecName: "kube-api-access-4j5b8") pod "3fc7afc5-4b83-45aa-9442-a75c500c32c5" (UID: "3fc7afc5-4b83-45aa-9442-a75c500c32c5"). InnerVolumeSpecName "kube-api-access-4j5b8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:28:31.492353 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.492329    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "3fc7afc5-4b83-45aa-9442-a75c500c32c5" (UID: "3fc7afc5-4b83-45aa-9442-a75c500c32c5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:28:31.495310 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.495285    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "3fc7afc5-4b83-45aa-9442-a75c500c32c5" (UID: "3fc7afc5-4b83-45aa-9442-a75c500c32c5"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:28:31.501749 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.501724    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-web-config" (OuterVolumeSpecName: "web-config") pod "3fc7afc5-4b83-45aa-9442-a75c500c32c5" (UID: "3fc7afc5-4b83-45aa-9442-a75c500c32c5"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:28:31.588609 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.588539    2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:28:31.588609 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.588569    2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fc7afc5-4b83-45aa-9442-a75c500c32c5-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:28:31.588609 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.588580    2568 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3fc7afc5-4b83-45aa-9442-a75c500c32c5-config-out\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:28:31.588609 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.588589    2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3fc7afc5-4b83-45aa-9442-a75c500c32c5-alertmanager-main-db\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:28:31.588609 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.588598    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4j5b8\" (UniqueName: \"kubernetes.io/projected/3fc7afc5-4b83-45aa-9442-a75c500c32c5-kube-api-access-4j5b8\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:28:31.588609 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.588607    2568 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-cluster-tls-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:28:31.588888 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.588615    2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:28:31.588888 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.588624    2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:28:31.588888 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.588632    2568 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3fc7afc5-4b83-45aa-9442-a75c500c32c5-tls-assets\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:28:31.588888 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.588640    2568 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-web-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:28:31.588888 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.588647    2568 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-config-volume\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:28:31.588888 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.588655    2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3fc7afc5-4b83-45aa-9442-a75c500c32c5-secret-alertmanager-main-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:28:31.939082 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.939050    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k6z9c" event={"ID":"bcf277d6-51cd-4bc8-ba3f-e6fcf272d363","Type":"ContainerStarted","Data":"743139f54c7f6a9b93a0a83d71d0d616010693e4f21d7860baa8ce4c908e78dc"}
Sep 18 06:28:31.942267 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.942241    2568 generic.go:356] "Generic (PLEG): container finished" podID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerID="30a51a79cd83f038afc473e0ed73ecf29b627e19d5779e1146ab41bdd47f171d" exitCode=0
Sep 18 06:28:31.942267 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.942261    2568 generic.go:356] "Generic (PLEG): container finished" podID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerID="3af7a4d343f51b4a24eceb49bc88a6a6086c0a7fa696d171ff71bf3286c761a1" exitCode=0
Sep 18 06:28:31.942446 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.942319    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc7afc5-4b83-45aa-9442-a75c500c32c5","Type":"ContainerDied","Data":"30a51a79cd83f038afc473e0ed73ecf29b627e19d5779e1146ab41bdd47f171d"}
Sep 18 06:28:31.942446 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.942351    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:31.942446 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.942388    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc7afc5-4b83-45aa-9442-a75c500c32c5","Type":"ContainerDied","Data":"3af7a4d343f51b4a24eceb49bc88a6a6086c0a7fa696d171ff71bf3286c761a1"}
Sep 18 06:28:31.942446 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.942417    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc7afc5-4b83-45aa-9442-a75c500c32c5","Type":"ContainerDied","Data":"d310b955b2aeacbad4d1da6637b8e7eee3ff89c327e664423f824da51f96541c"}
Sep 18 06:28:31.942662 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.942452    2568 scope.go:117] "RemoveContainer" containerID="d274c1f8fecd3e655bfdebbeb55be3a30259abccfecd88f799245fe69cef1b32"
Sep 18 06:28:31.958719 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.958706    2568 scope.go:117] "RemoveContainer" containerID="30a51a79cd83f038afc473e0ed73ecf29b627e19d5779e1146ab41bdd47f171d"
Sep 18 06:28:31.964944 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.964896    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 06:28:31.965496 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.965353    2568 scope.go:117] "RemoveContainer" containerID="2a9b265edcd5951a41ef029813295e2dba4e91982ff090a55f0fa4c1d4a2014b"
Sep 18 06:28:31.970003 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.969983    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 06:28:31.972513 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.972499    2568 scope.go:117] "RemoveContainer" containerID="3af7a4d343f51b4a24eceb49bc88a6a6086c0a7fa696d171ff71bf3286c761a1"
Sep 18 06:28:31.979673 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.979655    2568 scope.go:117] "RemoveContainer" containerID="a1272b1b065518f705e9bfa4ada9969df25e3b2e99a0a8d792279542ef74cea5"
Sep 18 06:28:31.987191 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.987161    2568 scope.go:117] "RemoveContainer" containerID="c9182d3c862f202444af58eb0a9e78157b673da5e841349b726cf93493689173"
Sep 18 06:28:31.994830 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.994808    2568 scope.go:117] "RemoveContainer" containerID="9cdcf778672dd29eb9d2f791e6de32a05d43bf3658e8baffa564bfe6e962732e"
Sep 18 06:28:31.996345 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.996325    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 06:28:31.996594 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.996581    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="kube-rbac-proxy-web"
Sep 18 06:28:31.996652 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.996596    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="kube-rbac-proxy-web"
Sep 18 06:28:31.996652 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.996607    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="kube-rbac-proxy-metric"
Sep 18 06:28:31.996652 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.996614    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="kube-rbac-proxy-metric"
Sep 18 06:28:31.996652 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.996627    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="config-reloader"
Sep 18 06:28:31.996652 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.996636    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="config-reloader"
Sep 18 06:28:31.996652 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.996650    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="alertmanager"
Sep 18 06:28:31.996938 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.996659    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="alertmanager"
Sep 18 06:28:31.996938 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.996671    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="kube-rbac-proxy"
Sep 18 06:28:31.996938 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.996679    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="kube-rbac-proxy"
Sep 18 06:28:31.996938 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.996688    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="prom-label-proxy"
Sep 18 06:28:31.996938 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.996696    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="prom-label-proxy"
Sep 18 06:28:31.996938 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.996707    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="init-config-reloader"
Sep 18 06:28:31.996938 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.996715    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="init-config-reloader"
Sep 18 06:28:31.996938 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.996772    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="config-reloader"
Sep 18 06:28:31.996938 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.996784    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="kube-rbac-proxy-web"
Sep 18 06:28:31.996938 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.996793    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="kube-rbac-proxy-metric"
Sep 18 06:28:31.996938 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.996802    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="alertmanager"
Sep 18 06:28:31.996938 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.996807    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="kube-rbac-proxy"
Sep 18 06:28:31.996938 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:31.996811    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" containerName="prom-label-proxy"
Sep 18 06:28:32.000020 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.000000    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.003649 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.003453    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 18 06:28:32.003649 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.003473    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 18 06:28:32.003794 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.003731    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 18 06:28:32.003850 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.003794    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qr5s8\""
Sep 18 06:28:32.004269 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.004059    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 18 06:28:32.004269 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.004061    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 18 06:28:32.004269 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.004136    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 18 06:28:32.004535 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.004365    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 18 06:28:32.004586 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.004558    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 18 06:28:32.004740 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.004717    2568 scope.go:117] "RemoveContainer" containerID="d274c1f8fecd3e655bfdebbeb55be3a30259abccfecd88f799245fe69cef1b32"
Sep 18 06:28:32.005521 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:28:32.005465    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d274c1f8fecd3e655bfdebbeb55be3a30259abccfecd88f799245fe69cef1b32\": container with ID starting with d274c1f8fecd3e655bfdebbeb55be3a30259abccfecd88f799245fe69cef1b32 not found: ID does not exist" containerID="d274c1f8fecd3e655bfdebbeb55be3a30259abccfecd88f799245fe69cef1b32"
Sep 18 06:28:32.005616 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.005539    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d274c1f8fecd3e655bfdebbeb55be3a30259abccfecd88f799245fe69cef1b32"} err="failed to get container status \"d274c1f8fecd3e655bfdebbeb55be3a30259abccfecd88f799245fe69cef1b32\": rpc error: code = NotFound desc = could not find container \"d274c1f8fecd3e655bfdebbeb55be3a30259abccfecd88f799245fe69cef1b32\": container with ID starting with d274c1f8fecd3e655bfdebbeb55be3a30259abccfecd88f799245fe69cef1b32 not found: ID does not exist"
Sep 18 06:28:32.005616 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.005563    2568 scope.go:117] "RemoveContainer" containerID="30a51a79cd83f038afc473e0ed73ecf29b627e19d5779e1146ab41bdd47f171d"
Sep 18 06:28:32.006175 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:28:32.006056    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30a51a79cd83f038afc473e0ed73ecf29b627e19d5779e1146ab41bdd47f171d\": container with ID starting with 30a51a79cd83f038afc473e0ed73ecf29b627e19d5779e1146ab41bdd47f171d not found: ID does not exist" containerID="30a51a79cd83f038afc473e0ed73ecf29b627e19d5779e1146ab41bdd47f171d"
Sep 18 06:28:32.006175 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.006087    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30a51a79cd83f038afc473e0ed73ecf29b627e19d5779e1146ab41bdd47f171d"} err="failed to get container status \"30a51a79cd83f038afc473e0ed73ecf29b627e19d5779e1146ab41bdd47f171d\": rpc error: code = NotFound desc = could not find container \"30a51a79cd83f038afc473e0ed73ecf29b627e19d5779e1146ab41bdd47f171d\": container with ID starting with 30a51a79cd83f038afc473e0ed73ecf29b627e19d5779e1146ab41bdd47f171d not found: ID does not exist"
Sep 18 06:28:32.006175 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.006109    2568 scope.go:117] "RemoveContainer" containerID="2a9b265edcd5951a41ef029813295e2dba4e91982ff090a55f0fa4c1d4a2014b"
Sep 18 06:28:32.006401 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:28:32.006386    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a9b265edcd5951a41ef029813295e2dba4e91982ff090a55f0fa4c1d4a2014b\": container with ID starting with 2a9b265edcd5951a41ef029813295e2dba4e91982ff090a55f0fa4c1d4a2014b not found: ID does not exist" containerID="2a9b265edcd5951a41ef029813295e2dba4e91982ff090a55f0fa4c1d4a2014b"
Sep 18 06:28:32.006471 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.006408    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a9b265edcd5951a41ef029813295e2dba4e91982ff090a55f0fa4c1d4a2014b"} err="failed to get container status \"2a9b265edcd5951a41ef029813295e2dba4e91982ff090a55f0fa4c1d4a2014b\": rpc error: code = NotFound desc = could not find container \"2a9b265edcd5951a41ef029813295e2dba4e91982ff090a55f0fa4c1d4a2014b\": container with ID starting with 2a9b265edcd5951a41ef029813295e2dba4e91982ff090a55f0fa4c1d4a2014b not found: ID does not exist"
Sep 18 06:28:32.006471 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.006423    2568 scope.go:117] "RemoveContainer" containerID="3af7a4d343f51b4a24eceb49bc88a6a6086c0a7fa696d171ff71bf3286c761a1"
Sep 18 06:28:32.006781 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:28:32.006658    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3af7a4d343f51b4a24eceb49bc88a6a6086c0a7fa696d171ff71bf3286c761a1\": container with ID starting with 3af7a4d343f51b4a24eceb49bc88a6a6086c0a7fa696d171ff71bf3286c761a1 not found: ID does not exist" containerID="3af7a4d343f51b4a24eceb49bc88a6a6086c0a7fa696d171ff71bf3286c761a1"
Sep 18 06:28:32.006781 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.006687    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3af7a4d343f51b4a24eceb49bc88a6a6086c0a7fa696d171ff71bf3286c761a1"} err="failed to get container status \"3af7a4d343f51b4a24eceb49bc88a6a6086c0a7fa696d171ff71bf3286c761a1\": rpc error: code = NotFound desc = could not find container \"3af7a4d343f51b4a24eceb49bc88a6a6086c0a7fa696d171ff71bf3286c761a1\": container with ID starting with 3af7a4d343f51b4a24eceb49bc88a6a6086c0a7fa696d171ff71bf3286c761a1 not found: ID does not exist"
Sep 18 06:28:32.006781 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.006708    2568 scope.go:117] "RemoveContainer" containerID="a1272b1b065518f705e9bfa4ada9969df25e3b2e99a0a8d792279542ef74cea5"
Sep 18 06:28:32.007266 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:28:32.007097    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1272b1b065518f705e9bfa4ada9969df25e3b2e99a0a8d792279542ef74cea5\": container with ID starting with a1272b1b065518f705e9bfa4ada9969df25e3b2e99a0a8d792279542ef74cea5 not found: ID does not exist" containerID="a1272b1b065518f705e9bfa4ada9969df25e3b2e99a0a8d792279542ef74cea5"
Sep 18 06:28:32.007266 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.007127    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1272b1b065518f705e9bfa4ada9969df25e3b2e99a0a8d792279542ef74cea5"} err="failed to get container status \"a1272b1b065518f705e9bfa4ada9969df25e3b2e99a0a8d792279542ef74cea5\": rpc error: code = NotFound desc = could not find container \"a1272b1b065518f705e9bfa4ada9969df25e3b2e99a0a8d792279542ef74cea5\": container with ID starting with a1272b1b065518f705e9bfa4ada9969df25e3b2e99a0a8d792279542ef74cea5 not found: ID does not exist"
Sep 18 06:28:32.007266 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.007147    2568 scope.go:117] "RemoveContainer" containerID="c9182d3c862f202444af58eb0a9e78157b673da5e841349b726cf93493689173"
Sep 18 06:28:32.007834 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:28:32.007769    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9182d3c862f202444af58eb0a9e78157b673da5e841349b726cf93493689173\": container with ID starting with c9182d3c862f202444af58eb0a9e78157b673da5e841349b726cf93493689173 not found: ID does not exist" containerID="c9182d3c862f202444af58eb0a9e78157b673da5e841349b726cf93493689173"
Sep 18 06:28:32.007834 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.007803    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9182d3c862f202444af58eb0a9e78157b673da5e841349b726cf93493689173"} err="failed to get container status \"c9182d3c862f202444af58eb0a9e78157b673da5e841349b726cf93493689173\": rpc error: code = NotFound desc = could not find container \"c9182d3c862f202444af58eb0a9e78157b673da5e841349b726cf93493689173\": container with ID starting with c9182d3c862f202444af58eb0a9e78157b673da5e841349b726cf93493689173 not found: ID does not exist"
Sep 18 06:28:32.007834 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.007823    2568 scope.go:117] "RemoveContainer" containerID="9cdcf778672dd29eb9d2f791e6de32a05d43bf3658e8baffa564bfe6e962732e"
Sep 18 06:28:32.008128 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:28:32.008108    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cdcf778672dd29eb9d2f791e6de32a05d43bf3658e8baffa564bfe6e962732e\": container with ID starting with 9cdcf778672dd29eb9d2f791e6de32a05d43bf3658e8baffa564bfe6e962732e not found: ID does not exist" containerID="9cdcf778672dd29eb9d2f791e6de32a05d43bf3658e8baffa564bfe6e962732e"
Sep 18 06:28:32.008195 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.008134    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cdcf778672dd29eb9d2f791e6de32a05d43bf3658e8baffa564bfe6e962732e"} err="failed to get container status \"9cdcf778672dd29eb9d2f791e6de32a05d43bf3658e8baffa564bfe6e962732e\": rpc error: code = NotFound desc = could not find container \"9cdcf778672dd29eb9d2f791e6de32a05d43bf3658e8baffa564bfe6e962732e\": container with ID starting with 9cdcf778672dd29eb9d2f791e6de32a05d43bf3658e8baffa564bfe6e962732e not found: ID does not exist"
Sep 18 06:28:32.008195 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.008154    2568 scope.go:117] "RemoveContainer" containerID="d274c1f8fecd3e655bfdebbeb55be3a30259abccfecd88f799245fe69cef1b32"
Sep 18 06:28:32.009238 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.009205    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 18 06:28:32.009334 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.009253    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d274c1f8fecd3e655bfdebbeb55be3a30259abccfecd88f799245fe69cef1b32"} err="failed to get container status \"d274c1f8fecd3e655bfdebbeb55be3a30259abccfecd88f799245fe69cef1b32\": rpc error: code = NotFound desc = could not find container \"d274c1f8fecd3e655bfdebbeb55be3a30259abccfecd88f799245fe69cef1b32\": container with ID starting with d274c1f8fecd3e655bfdebbeb55be3a30259abccfecd88f799245fe69cef1b32 not found: ID does not exist"
Sep 18 06:28:32.009334 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.009275    2568 scope.go:117] "RemoveContainer" containerID="30a51a79cd83f038afc473e0ed73ecf29b627e19d5779e1146ab41bdd47f171d"
Sep 18 06:28:32.009618 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.009592    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30a51a79cd83f038afc473e0ed73ecf29b627e19d5779e1146ab41bdd47f171d"} err="failed to get container status \"30a51a79cd83f038afc473e0ed73ecf29b627e19d5779e1146ab41bdd47f171d\": rpc error: code = NotFound desc = could not find container \"30a51a79cd83f038afc473e0ed73ecf29b627e19d5779e1146ab41bdd47f171d\": container with ID starting with 30a51a79cd83f038afc473e0ed73ecf29b627e19d5779e1146ab41bdd47f171d not found: ID does not exist"
Sep 18 06:28:32.009618 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.009615    2568 scope.go:117] "RemoveContainer" containerID="2a9b265edcd5951a41ef029813295e2dba4e91982ff090a55f0fa4c1d4a2014b"
Sep 18 06:28:32.009877 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.009858    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a9b265edcd5951a41ef029813295e2dba4e91982ff090a55f0fa4c1d4a2014b"} err="failed to get container status \"2a9b265edcd5951a41ef029813295e2dba4e91982ff090a55f0fa4c1d4a2014b\": rpc error: code = NotFound desc = could not find container \"2a9b265edcd5951a41ef029813295e2dba4e91982ff090a55f0fa4c1d4a2014b\": container with ID starting with 2a9b265edcd5951a41ef029813295e2dba4e91982ff090a55f0fa4c1d4a2014b not found: ID does not exist"
Sep 18 06:28:32.009942 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.009878    2568 scope.go:117] "RemoveContainer" containerID="3af7a4d343f51b4a24eceb49bc88a6a6086c0a7fa696d171ff71bf3286c761a1"
Sep 18 06:28:32.010117 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.010089    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3af7a4d343f51b4a24eceb49bc88a6a6086c0a7fa696d171ff71bf3286c761a1"} err="failed to get container status \"3af7a4d343f51b4a24eceb49bc88a6a6086c0a7fa696d171ff71bf3286c761a1\": rpc error: code = NotFound desc = could not find container \"3af7a4d343f51b4a24eceb49bc88a6a6086c0a7fa696d171ff71bf3286c761a1\": container with ID starting with 3af7a4d343f51b4a24eceb49bc88a6a6086c0a7fa696d171ff71bf3286c761a1 not found: ID does not exist"
Sep 18 06:28:32.010117 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.010116    2568 scope.go:117] "RemoveContainer" containerID="a1272b1b065518f705e9bfa4ada9969df25e3b2e99a0a8d792279542ef74cea5"
Sep 18 06:28:32.010442 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.010413    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1272b1b065518f705e9bfa4ada9969df25e3b2e99a0a8d792279542ef74cea5"} err="failed to get container status \"a1272b1b065518f705e9bfa4ada9969df25e3b2e99a0a8d792279542ef74cea5\": rpc error: code = NotFound desc = could not find container \"a1272b1b065518f705e9bfa4ada9969df25e3b2e99a0a8d792279542ef74cea5\": container with ID starting with a1272b1b065518f705e9bfa4ada9969df25e3b2e99a0a8d792279542ef74cea5 not found: ID does not exist"
Sep 18 06:28:32.010541 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.010441    2568 scope.go:117] "RemoveContainer" containerID="c9182d3c862f202444af58eb0a9e78157b673da5e841349b726cf93493689173"
Sep 18 06:28:32.010731 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.010703    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9182d3c862f202444af58eb0a9e78157b673da5e841349b726cf93493689173"} err="failed to get container status \"c9182d3c862f202444af58eb0a9e78157b673da5e841349b726cf93493689173\": rpc error: code = NotFound desc = could not find container \"c9182d3c862f202444af58eb0a9e78157b673da5e841349b726cf93493689173\": container with ID starting with c9182d3c862f202444af58eb0a9e78157b673da5e841349b726cf93493689173 not found: ID does not exist"
Sep 18 06:28:32.010806 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.010733    2568 scope.go:117] "RemoveContainer" containerID="9cdcf778672dd29eb9d2f791e6de32a05d43bf3658e8baffa564bfe6e962732e"
Sep 18 06:28:32.011007 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.010986    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cdcf778672dd29eb9d2f791e6de32a05d43bf3658e8baffa564bfe6e962732e"} err="failed to get container status \"9cdcf778672dd29eb9d2f791e6de32a05d43bf3658e8baffa564bfe6e962732e\": rpc error: code = NotFound desc = could not find container \"9cdcf778672dd29eb9d2f791e6de32a05d43bf3658e8baffa564bfe6e962732e\": container with ID starting with 9cdcf778672dd29eb9d2f791e6de32a05d43bf3658e8baffa564bfe6e962732e not found: ID does not exist"
Sep 18 06:28:32.017404 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.017383    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 06:28:32.093798 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.093767    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/623e7f44-b4e5-43df-99ed-b09621ccddf5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.093924 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.093816    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/623e7f44-b4e5-43df-99ed-b09621ccddf5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.093924 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.093849    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/623e7f44-b4e5-43df-99ed-b09621ccddf5-config-volume\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.093924 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.093908    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/623e7f44-b4e5-43df-99ed-b09621ccddf5-config-out\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.094085 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.093943    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/623e7f44-b4e5-43df-99ed-b09621ccddf5-web-config\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.094085 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.093972    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/623e7f44-b4e5-43df-99ed-b09621ccddf5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.094085 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.094002    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/623e7f44-b4e5-43df-99ed-b09621ccddf5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.094085 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.094029    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/623e7f44-b4e5-43df-99ed-b09621ccddf5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.094085 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.094065    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/623e7f44-b4e5-43df-99ed-b09621ccddf5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.094349 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.094123    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r82g\" (UniqueName: \"kubernetes.io/projected/623e7f44-b4e5-43df-99ed-b09621ccddf5-kube-api-access-4r82g\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.094349 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.094154    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/623e7f44-b4e5-43df-99ed-b09621ccddf5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.094349 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.094198    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/623e7f44-b4e5-43df-99ed-b09621ccddf5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.094349 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.094294    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/623e7f44-b4e5-43df-99ed-b09621ccddf5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.195431 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.195345    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/623e7f44-b4e5-43df-99ed-b09621ccddf5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.195431 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.195383    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/623e7f44-b4e5-43df-99ed-b09621ccddf5-config-volume\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.195431 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.195403    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/623e7f44-b4e5-43df-99ed-b09621ccddf5-config-out\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.195431 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.195419    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/623e7f44-b4e5-43df-99ed-b09621ccddf5-web-config\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.195733 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.195438    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/623e7f44-b4e5-43df-99ed-b09621ccddf5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.195733 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.195459    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/623e7f44-b4e5-43df-99ed-b09621ccddf5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.195733 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.195484    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/623e7f44-b4e5-43df-99ed-b09621ccddf5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.195733 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.195523    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/623e7f44-b4e5-43df-99ed-b09621ccddf5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.195733 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.195554    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r82g\" (UniqueName: \"kubernetes.io/projected/623e7f44-b4e5-43df-99ed-b09621ccddf5-kube-api-access-4r82g\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.195733 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.195586    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/623e7f44-b4e5-43df-99ed-b09621ccddf5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.195733 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.195612    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/623e7f44-b4e5-43df-99ed-b09621ccddf5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.195733 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.195649    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/623e7f44-b4e5-43df-99ed-b09621ccddf5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.195733 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.195699    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/623e7f44-b4e5-43df-99ed-b09621ccddf5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.196838 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.196495    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/623e7f44-b4e5-43df-99ed-b09621ccddf5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.196838 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.196566    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/623e7f44-b4e5-43df-99ed-b09621ccddf5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.196838 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.196778    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/623e7f44-b4e5-43df-99ed-b09621ccddf5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.198276 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.198251    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/623e7f44-b4e5-43df-99ed-b09621ccddf5-config-out\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.198603 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.198543    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/623e7f44-b4e5-43df-99ed-b09621ccddf5-web-config\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.198820 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.198797    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/623e7f44-b4e5-43df-99ed-b09621ccddf5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.199466 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.199081    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/623e7f44-b4e5-43df-99ed-b09621ccddf5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.199466 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.199136    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/623e7f44-b4e5-43df-99ed-b09621ccddf5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.199466 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.199312    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/623e7f44-b4e5-43df-99ed-b09621ccddf5-config-volume\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.199621 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.199480    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/623e7f44-b4e5-43df-99ed-b09621ccddf5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.200080 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.200055    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/623e7f44-b4e5-43df-99ed-b09621ccddf5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.200398 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.200383    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/623e7f44-b4e5-43df-99ed-b09621ccddf5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.203502 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.203479    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r82g\" (UniqueName: \"kubernetes.io/projected/623e7f44-b4e5-43df-99ed-b09621ccddf5-kube-api-access-4r82g\") pod \"alertmanager-main-0\" (UID: \"623e7f44-b4e5-43df-99ed-b09621ccddf5\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.312622 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.312594    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 18 06:28:32.541659 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.541582    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fc7afc5-4b83-45aa-9442-a75c500c32c5" path="/var/lib/kubelet/pods/3fc7afc5-4b83-45aa-9442-a75c500c32c5/volumes"
Sep 18 06:28:32.589564 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.589541    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 18 06:28:32.591795 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:28:32.591769    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod623e7f44_b4e5_43df_99ed_b09621ccddf5.slice/crio-4c944d753d54b5f1fbf5ee28e07339212d60a75b724d58468d606b0857f367fb WatchSource:0}: Error finding container 4c944d753d54b5f1fbf5ee28e07339212d60a75b724d58468d606b0857f367fb: Status 404 returned error can't find the container with id 4c944d753d54b5f1fbf5ee28e07339212d60a75b724d58468d606b0857f367fb
Sep 18 06:28:32.947201 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.947171    2568 generic.go:356] "Generic (PLEG): container finished" podID="623e7f44-b4e5-43df-99ed-b09621ccddf5" containerID="58f7a7a6de85d8755b3ca84bc92602c32dcb3559e32cfaa45525d346ef31b69a" exitCode=0
Sep 18 06:28:32.947486 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.947265    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"623e7f44-b4e5-43df-99ed-b09621ccddf5","Type":"ContainerDied","Data":"58f7a7a6de85d8755b3ca84bc92602c32dcb3559e32cfaa45525d346ef31b69a"}
Sep 18 06:28:32.947486 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.947356    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"623e7f44-b4e5-43df-99ed-b09621ccddf5","Type":"ContainerStarted","Data":"4c944d753d54b5f1fbf5ee28e07339212d60a75b724d58468d606b0857f367fb"}
Sep 18 06:28:32.949199 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.949177    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k6z9c" event={"ID":"bcf277d6-51cd-4bc8-ba3f-e6fcf272d363","Type":"ContainerStarted","Data":"6beae60c6459e8bac2e0468575c6ceb73cb9f24d21589311bcb6245d751b9644"}
Sep 18 06:28:32.949340 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.949235    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k6z9c" event={"ID":"bcf277d6-51cd-4bc8-ba3f-e6fcf272d363","Type":"ContainerStarted","Data":"6c0c46505e342fdb81bc661904dc7d793898fa09e90e0848bd4b3586cabc08e7"}
Sep 18 06:28:32.984396 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:32.984353    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-k6z9c" podStartSLOduration=376.042588243 podStartE2EDuration="6m16.984337838s" totalImagesPullingTime="941.749595ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:22:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:28:32.983273729 +0000 UTC m=+378.664801476" watchObservedRunningTime="2026-09-18 06:28:32.984337838 +0000 UTC m=+378.665865582"
Sep 18 06:28:33.067541 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:33.067518    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:33.164898 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:33.164873    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:33.957772 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:33.957729    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"623e7f44-b4e5-43df-99ed-b09621ccddf5","Type":"ContainerStarted","Data":"eef89ba7b3757743b25e3cf694933fbe37efb8a1dc0519c1aa142b32f24a3ea6"}
Sep 18 06:28:33.958186 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:33.957816    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"623e7f44-b4e5-43df-99ed-b09621ccddf5","Type":"ContainerStarted","Data":"09725b64a2c7b113c887d27ce0a42c151117c6ee17471bf0e9462a0b2e61514d"}
Sep 18 06:28:33.958186 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:33.957863    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"623e7f44-b4e5-43df-99ed-b09621ccddf5","Type":"ContainerStarted","Data":"e0da57a181b4c1b0bb625f365ee809c2550b16378bbb61f3d2c5c3f47f0be147"}
Sep 18 06:28:33.958186 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:33.957907    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"623e7f44-b4e5-43df-99ed-b09621ccddf5","Type":"ContainerStarted","Data":"0728a3a07a7c321743c5326ad0cb0ec771c81aeff7f5c8ea9fc454962d4baa39"}
Sep 18 06:28:33.958186 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:33.957944    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"623e7f44-b4e5-43df-99ed-b09621ccddf5","Type":"ContainerStarted","Data":"f6be76f748dfb687341a0f7b5b88886296b5217426ab247065f79b56f204bb65"}
Sep 18 06:28:33.958186 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:33.957979    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"623e7f44-b4e5-43df-99ed-b09621ccddf5","Type":"ContainerStarted","Data":"360e143eca5110f5083ba96bf4e75b7c173278da4ec8ac762b7e51815521f76c"}
Sep 18 06:28:33.983970 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:33.983925    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.968070289 podStartE2EDuration="2.983912779s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.84249ms" isStatefulPod=false podCreationTimestamp="2026-09-18 06:28:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:28:33.982493378 +0000 UTC m=+379.664021122" watchObservedRunningTime="2026-09-18 06:28:33.983912779 +0000 UTC m=+379.665440522"
Sep 18 06:28:34.127728 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.127690    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-886668cc9-5qcpz"]
Sep 18 06:28:34.129887 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.129866    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.132489 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.132469    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 18 06:28:34.132657 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.132648    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 18 06:28:34.132746 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.132702    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 18 06:28:34.132850 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.132835    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 18 06:28:34.132916 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.132896    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 18 06:28:34.133138 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.133114    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-v8xvg\""
Sep 18 06:28:34.137935 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.137912    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 18 06:28:34.144673 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.144645    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-886668cc9-5qcpz"]
Sep 18 06:28:34.213173 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.213095    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94e58ef4-4b8d-438b-91a3-f4b0be69234a-metrics-client-ca\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.213173 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.213144    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/94e58ef4-4b8d-438b-91a3-f4b0be69234a-secret-telemeter-client\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.213358 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.213208    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94e58ef4-4b8d-438b-91a3-f4b0be69234a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.213358 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.213260    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94e58ef4-4b8d-438b-91a3-f4b0be69234a-serving-certs-ca-bundle\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.213358 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.213277    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/94e58ef4-4b8d-438b-91a3-f4b0be69234a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.213358 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.213299    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/94e58ef4-4b8d-438b-91a3-f4b0be69234a-federate-client-tls\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.213536 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.213355    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rztjt\" (UniqueName: \"kubernetes.io/projected/94e58ef4-4b8d-438b-91a3-f4b0be69234a-kube-api-access-rztjt\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.213536 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.213405    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/94e58ef4-4b8d-438b-91a3-f4b0be69234a-telemeter-client-tls\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.313838 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.313805    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94e58ef4-4b8d-438b-91a3-f4b0be69234a-metrics-client-ca\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.313995 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.313845    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/94e58ef4-4b8d-438b-91a3-f4b0be69234a-secret-telemeter-client\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.313995 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.313867    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94e58ef4-4b8d-438b-91a3-f4b0be69234a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.313995 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.313888    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94e58ef4-4b8d-438b-91a3-f4b0be69234a-serving-certs-ca-bundle\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.313995 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.313915    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/94e58ef4-4b8d-438b-91a3-f4b0be69234a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.313995 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.313946    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/94e58ef4-4b8d-438b-91a3-f4b0be69234a-federate-client-tls\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.313995 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.313980    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rztjt\" (UniqueName: \"kubernetes.io/projected/94e58ef4-4b8d-438b-91a3-f4b0be69234a-kube-api-access-rztjt\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.314325 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.314022    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/94e58ef4-4b8d-438b-91a3-f4b0be69234a-telemeter-client-tls\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.314718 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.314689    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94e58ef4-4b8d-438b-91a3-f4b0be69234a-serving-certs-ca-bundle\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.314840 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.314689    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94e58ef4-4b8d-438b-91a3-f4b0be69234a-metrics-client-ca\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.314889 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.314836    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/94e58ef4-4b8d-438b-91a3-f4b0be69234a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.316765 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.316741    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/94e58ef4-4b8d-438b-91a3-f4b0be69234a-secret-telemeter-client\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.316832 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.316769    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/94e58ef4-4b8d-438b-91a3-f4b0be69234a-telemeter-client-tls\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.316877 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.316827    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/94e58ef4-4b8d-438b-91a3-f4b0be69234a-federate-client-tls\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.316912 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.316890    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/94e58ef4-4b8d-438b-91a3-f4b0be69234a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.321801 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.321784    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rztjt\" (UniqueName: \"kubernetes.io/projected/94e58ef4-4b8d-438b-91a3-f4b0be69234a-kube-api-access-rztjt\") pod \"telemeter-client-886668cc9-5qcpz\" (UID: \"94e58ef4-4b8d-438b-91a3-f4b0be69234a\") " pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.439621 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.439589    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz"
Sep 18 06:28:34.712106 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.712081    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-886668cc9-5qcpz"]
Sep 18 06:28:34.714729 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:28:34.714696    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94e58ef4_4b8d_438b_91a3_f4b0be69234a.slice/crio-caf63e29190ea2a20339201a26b3fd4b0087c2ac89d1a335d39e35d121d916e1 WatchSource:0}: Error finding container caf63e29190ea2a20339201a26b3fd4b0087c2ac89d1a335d39e35d121d916e1: Status 404 returned error can't find the container with id caf63e29190ea2a20339201a26b3fd4b0087c2ac89d1a335d39e35d121d916e1
Sep 18 06:28:34.963723 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:34.963685    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz" event={"ID":"94e58ef4-4b8d-438b-91a3-f4b0be69234a","Type":"ContainerStarted","Data":"caf63e29190ea2a20339201a26b3fd4b0087c2ac89d1a335d39e35d121d916e1"}
Sep 18 06:28:36.804095 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:36.804071    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:36.904104 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:36.904078    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:36.973912 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:36.973837    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz" event={"ID":"94e58ef4-4b8d-438b-91a3-f4b0be69234a","Type":"ContainerStarted","Data":"1c35aecdaf5a119ae750d4d86f096a7b26a5f43afb741cf0414ed158dd90b5ee"}
Sep 18 06:28:36.973912 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:36.973904    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz" event={"ID":"94e58ef4-4b8d-438b-91a3-f4b0be69234a","Type":"ContainerStarted","Data":"022c08d20f88ef3434a772e67586b7e94c758ad11be6602cca1bcd92764ebc1b"}
Sep 18 06:28:36.974068 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:36.973933    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz" event={"ID":"94e58ef4-4b8d-438b-91a3-f4b0be69234a","Type":"ContainerStarted","Data":"1d05f31c2d7aadf0534f46aebaa8b6a299450e9a9c15193d861031f1aaaadd40"}
Sep 18 06:28:36.995080 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:36.995041    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-886668cc9-5qcpz" podStartSLOduration=1.026891739 podStartE2EDuration="2.995027591s" totalImagesPullingTime="1.968135852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:28:36.994186897 +0000 UTC m=+382.675714641" watchObservedRunningTime="2026-09-18 06:28:36.995027591 +0000 UTC m=+382.676555335"
Sep 18 06:28:37.121983 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:37.121950    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:37.222391 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:37.222362    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:37.322057 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:37.321979    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:37.871131 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:37.871098    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:37.919186 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:37.919154    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:47.793422 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:47.793387    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fbbd7b6d6-n25dt"]
Sep 18 06:28:47.892146 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:47.892116    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:47.941940 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:47.941917    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:47.995584 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:47.995562    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:48.096065 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:48.095994    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:49.470137 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:49.470105    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:28:49.525663 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:28:49.525633    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:29:07.153667 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:07.153636    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:29:07.204071 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:07.204039    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:29:12.812046 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:12.812004    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5fbbd7b6d6-n25dt" podUID="5c8ed825-382d-4cbe-9342-f1ce600bed96" containerName="console" containerID="cri-o://9ea77edc37d3027382b3c65dc8ee44338f2b8122b3225ea0a1547d34b2778ab4" gracePeriod=15
Sep 18 06:29:13.054778 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.054749    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:29:13.081921 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.081858    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c8ed825-382d-4cbe-9342-f1ce600bed96-console-oauth-config\") pod \"5c8ed825-382d-4cbe-9342-f1ce600bed96\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") "
Sep 18 06:29:13.081921 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.081904    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-trusted-ca-bundle\") pod \"5c8ed825-382d-4cbe-9342-f1ce600bed96\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") "
Sep 18 06:29:13.082109 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.081926    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qbcc2\" (UniqueName: \"kubernetes.io/projected/5c8ed825-382d-4cbe-9342-f1ce600bed96-kube-api-access-qbcc2\") pod \"5c8ed825-382d-4cbe-9342-f1ce600bed96\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") "
Sep 18 06:29:13.082109 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.081953    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-oauth-serving-cert\") pod \"5c8ed825-382d-4cbe-9342-f1ce600bed96\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") "
Sep 18 06:29:13.082109 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.081987    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c8ed825-382d-4cbe-9342-f1ce600bed96-console-serving-cert\") pod \"5c8ed825-382d-4cbe-9342-f1ce600bed96\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") "
Sep 18 06:29:13.082109 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.082022    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-console-config\") pod \"5c8ed825-382d-4cbe-9342-f1ce600bed96\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") "
Sep 18 06:29:13.082109 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.082058    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-service-ca\") pod \"5c8ed825-382d-4cbe-9342-f1ce600bed96\" (UID: \"5c8ed825-382d-4cbe-9342-f1ce600bed96\") "
Sep 18 06:29:13.082390 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.082348    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5c8ed825-382d-4cbe-9342-f1ce600bed96" (UID: "5c8ed825-382d-4cbe-9342-f1ce600bed96"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 06:29:13.082540 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.082416    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5c8ed825-382d-4cbe-9342-f1ce600bed96" (UID: "5c8ed825-382d-4cbe-9342-f1ce600bed96"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 06:29:13.082645 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.082578    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-console-config" (OuterVolumeSpecName: "console-config") pod "5c8ed825-382d-4cbe-9342-f1ce600bed96" (UID: "5c8ed825-382d-4cbe-9342-f1ce600bed96"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 06:29:13.082645 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.082585    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-service-ca" (OuterVolumeSpecName: "service-ca") pod "5c8ed825-382d-4cbe-9342-f1ce600bed96" (UID: "5c8ed825-382d-4cbe-9342-f1ce600bed96"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 06:29:13.084067 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.084046    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c8ed825-382d-4cbe-9342-f1ce600bed96-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5c8ed825-382d-4cbe-9342-f1ce600bed96" (UID: "5c8ed825-382d-4cbe-9342-f1ce600bed96"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:29:13.084170 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.084153    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c8ed825-382d-4cbe-9342-f1ce600bed96-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5c8ed825-382d-4cbe-9342-f1ce600bed96" (UID: "5c8ed825-382d-4cbe-9342-f1ce600bed96"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:29:13.084539 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.084518    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c8ed825-382d-4cbe-9342-f1ce600bed96-kube-api-access-qbcc2" (OuterVolumeSpecName: "kube-api-access-qbcc2") pod "5c8ed825-382d-4cbe-9342-f1ce600bed96" (UID: "5c8ed825-382d-4cbe-9342-f1ce600bed96"). InnerVolumeSpecName "kube-api-access-qbcc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:29:13.104083 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.104056    2568 generic.go:356] "Generic (PLEG): container finished" podID="5c8ed825-382d-4cbe-9342-f1ce600bed96" containerID="9ea77edc37d3027382b3c65dc8ee44338f2b8122b3225ea0a1547d34b2778ab4" exitCode=0
Sep 18 06:29:13.104184 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.104098    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5fbbd7b6d6-n25dt"
Sep 18 06:29:13.104184 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.104137    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fbbd7b6d6-n25dt" event={"ID":"5c8ed825-382d-4cbe-9342-f1ce600bed96","Type":"ContainerDied","Data":"9ea77edc37d3027382b3c65dc8ee44338f2b8122b3225ea0a1547d34b2778ab4"}
Sep 18 06:29:13.104323 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.104195    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5fbbd7b6d6-n25dt" event={"ID":"5c8ed825-382d-4cbe-9342-f1ce600bed96","Type":"ContainerDied","Data":"8380c286d865ba5c19298d478c89da2d494cff18f9f9825c5a0000169dcb419d"}
Sep 18 06:29:13.104323 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.104241    2568 scope.go:117] "RemoveContainer" containerID="9ea77edc37d3027382b3c65dc8ee44338f2b8122b3225ea0a1547d34b2778ab4"
Sep 18 06:29:13.112796 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.112776    2568 scope.go:117] "RemoveContainer" containerID="9ea77edc37d3027382b3c65dc8ee44338f2b8122b3225ea0a1547d34b2778ab4"
Sep 18 06:29:13.113051 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:29:13.113034    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ea77edc37d3027382b3c65dc8ee44338f2b8122b3225ea0a1547d34b2778ab4\": container with ID starting with 9ea77edc37d3027382b3c65dc8ee44338f2b8122b3225ea0a1547d34b2778ab4 not found: ID does not exist" containerID="9ea77edc37d3027382b3c65dc8ee44338f2b8122b3225ea0a1547d34b2778ab4"
Sep 18 06:29:13.113095 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.113060    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ea77edc37d3027382b3c65dc8ee44338f2b8122b3225ea0a1547d34b2778ab4"} err="failed to get container status \"9ea77edc37d3027382b3c65dc8ee44338f2b8122b3225ea0a1547d34b2778ab4\": rpc error: code = NotFound desc = could not find container \"9ea77edc37d3027382b3c65dc8ee44338f2b8122b3225ea0a1547d34b2778ab4\": container with ID starting with 9ea77edc37d3027382b3c65dc8ee44338f2b8122b3225ea0a1547d34b2778ab4 not found: ID does not exist"
Sep 18 06:29:13.125005 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.124985    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5fbbd7b6d6-n25dt"]
Sep 18 06:29:13.128405 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.128388    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5fbbd7b6d6-n25dt"]
Sep 18 06:29:13.183267 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.183242    2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c8ed825-382d-4cbe-9342-f1ce600bed96-console-oauth-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:29:13.183267 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.183262    2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-trusted-ca-bundle\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:29:13.183396 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.183272    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbcc2\" (UniqueName: \"kubernetes.io/projected/5c8ed825-382d-4cbe-9342-f1ce600bed96-kube-api-access-qbcc2\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:29:13.183396 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.183283    2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-oauth-serving-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:29:13.183396 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.183291    2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c8ed825-382d-4cbe-9342-f1ce600bed96-console-serving-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:29:13.183396 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.183300    2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-console-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:29:13.183396 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:13.183308    2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c8ed825-382d-4cbe-9342-f1ce600bed96-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:29:14.542116 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:14.542082    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c8ed825-382d-4cbe-9342-f1ce600bed96" path="/var/lib/kubelet/pods/5c8ed825-382d-4cbe-9342-f1ce600bed96/volumes"
Sep 18 06:29:44.294189 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:44.294154    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:29:44.344792 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:44.344761    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:29:56.295379 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:56.295346    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:29:56.346523 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:29:56.346492    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:30:00.320329 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.320293    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b87fd654d-v9swj"]
Sep 18 06:30:00.320710 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.320578    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c8ed825-382d-4cbe-9342-f1ce600bed96" containerName="console"
Sep 18 06:30:00.320710 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.320591    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c8ed825-382d-4cbe-9342-f1ce600bed96" containerName="console"
Sep 18 06:30:00.320710 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.320646    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c8ed825-382d-4cbe-9342-f1ce600bed96" containerName="console"
Sep 18 06:30:00.323428 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.323409    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.326943 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.326921    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 18 06:30:00.327067 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.327040    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 18 06:30:00.327067 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.326921    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 18 06:30:00.327180 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.326941    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 18 06:30:00.327180 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.326958    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 18 06:30:00.327180 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.326958    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-q62qv\""
Sep 18 06:30:00.331319 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.331300    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 18 06:30:00.334904 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.334884    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b87fd654d-v9swj"]
Sep 18 06:30:00.380467 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.380429    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:30:00.430528 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.430497    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-console-config\") pod \"console-5b87fd654d-v9swj\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") " pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.430528 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.430539    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-trusted-ca-bundle\") pod \"console-5b87fd654d-v9swj\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") " pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.430740 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.430593    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-oauth-serving-cert\") pod \"console-5b87fd654d-v9swj\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") " pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.430740 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.430632    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9907de51-d4f8-4067-ad96-1a1e5e3448ad-console-serving-cert\") pod \"console-5b87fd654d-v9swj\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") " pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.430740 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.430688    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9907de51-d4f8-4067-ad96-1a1e5e3448ad-console-oauth-config\") pod \"console-5b87fd654d-v9swj\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") " pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.430740 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.430709    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-service-ca\") pod \"console-5b87fd654d-v9swj\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") " pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.430740 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.430725    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lm5h\" (UniqueName: \"kubernetes.io/projected/9907de51-d4f8-4067-ad96-1a1e5e3448ad-kube-api-access-7lm5h\") pod \"console-5b87fd654d-v9swj\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") " pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.433864 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.433837    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:30:00.521920 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.521890    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:30:00.531992 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.531966    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-console-config\") pod \"console-5b87fd654d-v9swj\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") " pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.532088 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.532003    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-trusted-ca-bundle\") pod \"console-5b87fd654d-v9swj\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") " pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.532088 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.532032    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-oauth-serving-cert\") pod \"console-5b87fd654d-v9swj\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") " pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.532088 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.532056    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9907de51-d4f8-4067-ad96-1a1e5e3448ad-console-serving-cert\") pod \"console-5b87fd654d-v9swj\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") " pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.532244 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.532091    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9907de51-d4f8-4067-ad96-1a1e5e3448ad-console-oauth-config\") pod \"console-5b87fd654d-v9swj\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") " pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.532244 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.532109    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-service-ca\") pod \"console-5b87fd654d-v9swj\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") " pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.532244 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.532123    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lm5h\" (UniqueName: \"kubernetes.io/projected/9907de51-d4f8-4067-ad96-1a1e5e3448ad-kube-api-access-7lm5h\") pod \"console-5b87fd654d-v9swj\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") " pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.532831 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.532805    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-console-config\") pod \"console-5b87fd654d-v9swj\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") " pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.532954 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.532912    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-service-ca\") pod \"console-5b87fd654d-v9swj\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") " pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.533014 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.532955    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-oauth-serving-cert\") pod \"console-5b87fd654d-v9swj\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") " pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.533167 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.533151    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-trusted-ca-bundle\") pod \"console-5b87fd654d-v9swj\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") " pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.535066 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.535039    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9907de51-d4f8-4067-ad96-1a1e5e3448ad-console-serving-cert\") pod \"console-5b87fd654d-v9swj\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") " pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.535172 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.535153    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9907de51-d4f8-4067-ad96-1a1e5e3448ad-console-oauth-config\") pod \"console-5b87fd654d-v9swj\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") " pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.539649 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.539632    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lm5h\" (UniqueName: \"kubernetes.io/projected/9907de51-d4f8-4067-ad96-1a1e5e3448ad-kube-api-access-7lm5h\") pod \"console-5b87fd654d-v9swj\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") " pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.574174 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.574089    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:30:00.633291 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.633249    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:00.903740 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:00.903716    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b87fd654d-v9swj"]
Sep 18 06:30:00.906040 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:30:00.906012    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9907de51_d4f8_4067_ad96_1a1e5e3448ad.slice/crio-0fe3b79ab9fc8487f06ad1760769fc610d84cd6d73fe155b1bcfb25a4073b9d8 WatchSource:0}: Error finding container 0fe3b79ab9fc8487f06ad1760769fc610d84cd6d73fe155b1bcfb25a4073b9d8: Status 404 returned error can't find the container with id 0fe3b79ab9fc8487f06ad1760769fc610d84cd6d73fe155b1bcfb25a4073b9d8
Sep 18 06:30:01.121970 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:01.121935    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:30:01.267760 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:01.267675    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:30:01.278042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:01.278005    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b87fd654d-v9swj" event={"ID":"9907de51-d4f8-4067-ad96-1a1e5e3448ad","Type":"ContainerStarted","Data":"84b5c06db093703b3f13b12fd24abe908f85573538fc4d422f6b3e76af220491"}
Sep 18 06:30:01.278193 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:01.278052    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b87fd654d-v9swj" event={"ID":"9907de51-d4f8-4067-ad96-1a1e5e3448ad","Type":"ContainerStarted","Data":"0fe3b79ab9fc8487f06ad1760769fc610d84cd6d73fe155b1bcfb25a4073b9d8"}
Sep 18 06:30:01.294962 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:01.294917    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b87fd654d-v9swj" podStartSLOduration=1.294901516 podStartE2EDuration="1.294901516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:30:01.293681856 +0000 UTC m=+466.975209600" watchObservedRunningTime="2026-09-18 06:30:01.294901516 +0000 UTC m=+466.976429259"
Sep 18 06:30:06.435824 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:06.435779    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:30:06.534463 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:06.534432    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:30:07.108910 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:07.108882    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:30:10.633763 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:10.633713    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:10.633763 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:10.633783    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:10.639201 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:10.639176    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:11.317658 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:11.317625    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:30:11.476855 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:11.476822    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:30:11.521975 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:11.521952    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:30:11.576984 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:11.576929    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:30:52.717118 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:52.717080    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:30:52.814180 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:30:52.814143    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:31:16.387812 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:16.387776    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:31:19.306550 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:19.306511    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:31:20.091562 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:20.091527    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:31:22.920331 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:22.920299    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:31:22.970232 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:22.970188    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:31:24.717119 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:24.717092    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:31:24.833187 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:24.833155    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:31:24.943235 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:24.943189    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:31:26.507251 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:26.507211    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:31:27.094298 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:27.094265    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:31:29.719454 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:29.719426    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:31:29.770055 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:29.770024    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:31:32.906173 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:32.906142    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:31:32.954085 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:32.954051    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:31:35.899651 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:35.899618    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:31:37.388910 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:37.388875    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:31:37.483659 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:37.483622    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:31:43.575735 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:43.575698    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:31:43.675955 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:43.675919    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:31:45.308860 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:45.308830    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:31:45.412511 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:45.412473    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rtccx"]
Sep 18 06:31:45.415441 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:45.415427    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rtccx"
Sep 18 06:31:45.417894 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:45.417861    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 18 06:31:45.418019 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:45.417946    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 18 06:31:45.418075 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:45.418016    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-2ddh6\""
Sep 18 06:31:45.425288 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:45.425265    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xltrl\" (UniqueName: \"kubernetes.io/projected/423e2262-72d7-4f76-984c-b16918a3592c-kube-api-access-xltrl\") pod \"cert-manager-55b4d954d5-rtccx\" (UID: \"423e2262-72d7-4f76-984c-b16918a3592c\") " pod="cert-manager/cert-manager-55b4d954d5-rtccx"
Sep 18 06:31:45.425400 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:45.425299    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/423e2262-72d7-4f76-984c-b16918a3592c-bound-sa-token\") pod \"cert-manager-55b4d954d5-rtccx\" (UID: \"423e2262-72d7-4f76-984c-b16918a3592c\") " pod="cert-manager/cert-manager-55b4d954d5-rtccx"
Sep 18 06:31:45.425444 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:45.425397    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rtccx"]
Sep 18 06:31:45.526389 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:45.526359    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xltrl\" (UniqueName: \"kubernetes.io/projected/423e2262-72d7-4f76-984c-b16918a3592c-kube-api-access-xltrl\") pod \"cert-manager-55b4d954d5-rtccx\" (UID: \"423e2262-72d7-4f76-984c-b16918a3592c\") " pod="cert-manager/cert-manager-55b4d954d5-rtccx"
Sep 18 06:31:45.526389 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:45.526401    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/423e2262-72d7-4f76-984c-b16918a3592c-bound-sa-token\") pod \"cert-manager-55b4d954d5-rtccx\" (UID: \"423e2262-72d7-4f76-984c-b16918a3592c\") " pod="cert-manager/cert-manager-55b4d954d5-rtccx"
Sep 18 06:31:45.534652 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:45.534615    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/423e2262-72d7-4f76-984c-b16918a3592c-bound-sa-token\") pod \"cert-manager-55b4d954d5-rtccx\" (UID: \"423e2262-72d7-4f76-984c-b16918a3592c\") " pod="cert-manager/cert-manager-55b4d954d5-rtccx"
Sep 18 06:31:45.534774 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:45.534697    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xltrl\" (UniqueName: \"kubernetes.io/projected/423e2262-72d7-4f76-984c-b16918a3592c-kube-api-access-xltrl\") pod \"cert-manager-55b4d954d5-rtccx\" (UID: \"423e2262-72d7-4f76-984c-b16918a3592c\") " pod="cert-manager/cert-manager-55b4d954d5-rtccx"
Sep 18 06:31:45.724699 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:45.724664    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-rtccx"
Sep 18 06:31:45.992397 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:45.992280    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-rtccx"]
Sep 18 06:31:45.994905 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:31:45.994879    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod423e2262_72d7_4f76_984c_b16918a3592c.slice/crio-1f11815e4037ae26da8874e99d88ee1485ad645f101a71025b5e1aa9eca19f5c WatchSource:0}: Error finding container 1f11815e4037ae26da8874e99d88ee1485ad645f101a71025b5e1aa9eca19f5c: Status 404 returned error can't find the container with id 1f11815e4037ae26da8874e99d88ee1485ad645f101a71025b5e1aa9eca19f5c
Sep 18 06:31:46.651636 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:46.651585    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rtccx" event={"ID":"423e2262-72d7-4f76-984c-b16918a3592c","Type":"ContainerStarted","Data":"1f11815e4037ae26da8874e99d88ee1485ad645f101a71025b5e1aa9eca19f5c"}
Sep 18 06:31:49.667470 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:49.667432    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-rtccx" event={"ID":"423e2262-72d7-4f76-984c-b16918a3592c","Type":"ContainerStarted","Data":"1be5b76c0294c515064b54d24ce655baad9593345b626236bf693a4df520638b"}
Sep 18 06:31:49.682974 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:49.682921    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-rtccx" podStartSLOduration=1.710616192 podStartE2EDuration="4.682902654s" totalImagesPullingTime="2.972286462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:31:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:31:49.682728269 +0000 UTC m=+575.364256010" watchObservedRunningTime="2026-09-18 06:31:49.682902654 +0000 UTC m=+575.364430399"
Sep 18 06:31:53.516937 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:53.516905    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:31:53.603828 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:31:53.603796    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:01.351052 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:01.351010    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:01.462175 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:01.462138    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:01.997754 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:01.997721    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:02.252616 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.252521    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:02.350167 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.350132    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:02.390983 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.390952    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk"]
Sep 18 06:32:02.394297 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.394278    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk"
Sep 18 06:32:02.396818 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.396801    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 18 06:32:02.396938 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.396819    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 18 06:32:02.397063 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.397048    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zlgzp\""
Sep 18 06:32:02.397154 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.397110    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 18 06:32:02.397212 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.397155    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 18 06:32:02.410445 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.410424    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk"]
Sep 18 06:32:02.453890 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.453858    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/54caad4e-0c6a-43f8-bdc3-eabffaaf7c19-manifests\") pod \"opendatahub-operator-controller-manager-6fd575c64f-xv5qk\" (UID: \"54caad4e-0c6a-43f8-bdc3-eabffaaf7c19\") " pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk"
Sep 18 06:32:02.453890 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.453890    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4prvt\" (UniqueName: \"kubernetes.io/projected/54caad4e-0c6a-43f8-bdc3-eabffaaf7c19-kube-api-access-4prvt\") pod \"opendatahub-operator-controller-manager-6fd575c64f-xv5qk\" (UID: \"54caad4e-0c6a-43f8-bdc3-eabffaaf7c19\") " pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk"
Sep 18 06:32:02.454098 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.453916    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/54caad4e-0c6a-43f8-bdc3-eabffaaf7c19-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fd575c64f-xv5qk\" (UID: \"54caad4e-0c6a-43f8-bdc3-eabffaaf7c19\") " pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk"
Sep 18 06:32:02.454098 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.454016    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/54caad4e-0c6a-43f8-bdc3-eabffaaf7c19-tmp\") pod \"opendatahub-operator-controller-manager-6fd575c64f-xv5qk\" (UID: \"54caad4e-0c6a-43f8-bdc3-eabffaaf7c19\") " pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk"
Sep 18 06:32:02.454098 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.454040    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/54caad4e-0c6a-43f8-bdc3-eabffaaf7c19-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fd575c64f-xv5qk\" (UID: \"54caad4e-0c6a-43f8-bdc3-eabffaaf7c19\") " pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk"
Sep 18 06:32:02.517002 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.516922    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:02.554934 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.554899    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/54caad4e-0c6a-43f8-bdc3-eabffaaf7c19-manifests\") pod \"opendatahub-operator-controller-manager-6fd575c64f-xv5qk\" (UID: \"54caad4e-0c6a-43f8-bdc3-eabffaaf7c19\") " pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk"
Sep 18 06:32:02.554934 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.554937    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4prvt\" (UniqueName: \"kubernetes.io/projected/54caad4e-0c6a-43f8-bdc3-eabffaaf7c19-kube-api-access-4prvt\") pod \"opendatahub-operator-controller-manager-6fd575c64f-xv5qk\" (UID: \"54caad4e-0c6a-43f8-bdc3-eabffaaf7c19\") " pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk"
Sep 18 06:32:02.555151 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.554974    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/54caad4e-0c6a-43f8-bdc3-eabffaaf7c19-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fd575c64f-xv5qk\" (UID: \"54caad4e-0c6a-43f8-bdc3-eabffaaf7c19\") " pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk"
Sep 18 06:32:02.555151 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.555047    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/54caad4e-0c6a-43f8-bdc3-eabffaaf7c19-tmp\") pod \"opendatahub-operator-controller-manager-6fd575c64f-xv5qk\" (UID: \"54caad4e-0c6a-43f8-bdc3-eabffaaf7c19\") " pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk"
Sep 18 06:32:02.555151 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.555073    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/54caad4e-0c6a-43f8-bdc3-eabffaaf7c19-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fd575c64f-xv5qk\" (UID: \"54caad4e-0c6a-43f8-bdc3-eabffaaf7c19\") " pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk"
Sep 18 06:32:02.555400 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.555297    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/54caad4e-0c6a-43f8-bdc3-eabffaaf7c19-manifests\") pod \"opendatahub-operator-controller-manager-6fd575c64f-xv5qk\" (UID: \"54caad4e-0c6a-43f8-bdc3-eabffaaf7c19\") " pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk"
Sep 18 06:32:02.557396 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.557363    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/54caad4e-0c6a-43f8-bdc3-eabffaaf7c19-tmp\") pod \"opendatahub-operator-controller-manager-6fd575c64f-xv5qk\" (UID: \"54caad4e-0c6a-43f8-bdc3-eabffaaf7c19\") " pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk"
Sep 18 06:32:02.557514 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.557432    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/54caad4e-0c6a-43f8-bdc3-eabffaaf7c19-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6fd575c64f-xv5qk\" (UID: \"54caad4e-0c6a-43f8-bdc3-eabffaaf7c19\") " pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk"
Sep 18 06:32:02.557568 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.557510    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/54caad4e-0c6a-43f8-bdc3-eabffaaf7c19-webhook-cert\") pod \"opendatahub-operator-controller-manager-6fd575c64f-xv5qk\" (UID: \"54caad4e-0c6a-43f8-bdc3-eabffaaf7c19\") " pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk"
Sep 18 06:32:02.562731 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.562710    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4prvt\" (UniqueName: \"kubernetes.io/projected/54caad4e-0c6a-43f8-bdc3-eabffaaf7c19-kube-api-access-4prvt\") pod \"opendatahub-operator-controller-manager-6fd575c64f-xv5qk\" (UID: \"54caad4e-0c6a-43f8-bdc3-eabffaaf7c19\") " pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk"
Sep 18 06:32:02.604377 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.604348    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:02.704350 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.704318    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk"
Sep 18 06:32:02.705583 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.705559    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:02.817568 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.817493    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:02.914549 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.914516    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:02.977875 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.977855    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk"]
Sep 18 06:32:02.979574 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:32:02.979548    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54caad4e_0c6a_43f8_bdc3_eabffaaf7c19.slice/crio-541854b023c941eed7b05dd6ec0227fe391a3813825edabc317206d1244eac0c WatchSource:0}: Error finding container 541854b023c941eed7b05dd6ec0227fe391a3813825edabc317206d1244eac0c: Status 404 returned error can't find the container with id 541854b023c941eed7b05dd6ec0227fe391a3813825edabc317206d1244eac0c
Sep 18 06:32:02.988681 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:02.988658    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:03.056548 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:03.056517    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:03.722660 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:03.722616    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk" event={"ID":"54caad4e-0c6a-43f8-bdc3-eabffaaf7c19","Type":"ContainerStarted","Data":"541854b023c941eed7b05dd6ec0227fe391a3813825edabc317206d1244eac0c"}
Sep 18 06:32:06.738628 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:06.738589    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk" event={"ID":"54caad4e-0c6a-43f8-bdc3-eabffaaf7c19","Type":"ContainerStarted","Data":"5137d2d1f598a3852a88c3a237c233c5f85a0c0d69c120b6f664d189ac3075cc"}
Sep 18 06:32:07.743494 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:07.743461    2568 generic.go:356] "Generic (PLEG): container finished" podID="54caad4e-0c6a-43f8-bdc3-eabffaaf7c19" containerID="5137d2d1f598a3852a88c3a237c233c5f85a0c0d69c120b6f664d189ac3075cc" exitCode=0
Sep 18 06:32:07.743888 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:07.743532    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk" event={"ID":"54caad4e-0c6a-43f8-bdc3-eabffaaf7c19","Type":"ContainerDied","Data":"5137d2d1f598a3852a88c3a237c233c5f85a0c0d69c120b6f664d189ac3075cc"}
Sep 18 06:32:08.750315 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:08.750200    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk" event={"ID":"54caad4e-0c6a-43f8-bdc3-eabffaaf7c19","Type":"ContainerStarted","Data":"5930b0142a24b9884fe071b05efd0f60dbcd43a04f382165bafc44066a791b6d"}
Sep 18 06:32:08.750751 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:08.750339    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk"
Sep 18 06:32:08.770521 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:08.770469    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk" podStartSLOduration=2.274681719 podStartE2EDuration="6.770453133s" totalImagesPullingTime="3.490385145s" totalInitContainerRuntime="1.005386269s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:32:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:32:08.769043699 +0000 UTC m=+594.450571454" watchObservedRunningTime="2026-09-18 06:32:08.770453133 +0000 UTC m=+594.451980876"
Sep 18 06:32:14.103427 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:14.103390    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:15.387889 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:15.387858    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:16.352977 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:16.352944    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:16.403861 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:16.403832    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:19.756825 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:19.756793    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6fd575c64f-xv5qk"
Sep 18 06:32:21.647440 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:21.647405    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:21.736550 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:21.736515    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:21.995056 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:21.994967    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:22.033982 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:22.033950    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5bbd68dc59-6grwd"]
Sep 18 06:32:22.040870 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:22.040844    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5bbd68dc59-6grwd"
Sep 18 06:32:22.043563 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:22.043539    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 18 06:32:22.044607 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:22.044591    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-tkjkp\""
Sep 18 06:32:22.044738 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:22.044711    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 18 06:32:22.044843 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:22.044745    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 18 06:32:22.045162 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:22.045106    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 18 06:32:22.046838 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:22.046815    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5bbd68dc59-6grwd"]
Sep 18 06:32:22.091516 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:22.091475    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:22.113641 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:22.113611    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/415ce0db-109b-4944-b4aa-2ed7733a8b0e-tls-certs\") pod \"kube-auth-proxy-5bbd68dc59-6grwd\" (UID: \"415ce0db-109b-4944-b4aa-2ed7733a8b0e\") " pod="openshift-ingress/kube-auth-proxy-5bbd68dc59-6grwd"
Sep 18 06:32:22.113790 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:22.113647    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwcwq\" (UniqueName: \"kubernetes.io/projected/415ce0db-109b-4944-b4aa-2ed7733a8b0e-kube-api-access-cwcwq\") pod \"kube-auth-proxy-5bbd68dc59-6grwd\" (UID: \"415ce0db-109b-4944-b4aa-2ed7733a8b0e\") " pod="openshift-ingress/kube-auth-proxy-5bbd68dc59-6grwd"
Sep 18 06:32:22.113790 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:22.113685    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/415ce0db-109b-4944-b4aa-2ed7733a8b0e-tmp\") pod \"kube-auth-proxy-5bbd68dc59-6grwd\" (UID: \"415ce0db-109b-4944-b4aa-2ed7733a8b0e\") " pod="openshift-ingress/kube-auth-proxy-5bbd68dc59-6grwd"
Sep 18 06:32:22.214560 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:22.214528    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwcwq\" (UniqueName: \"kubernetes.io/projected/415ce0db-109b-4944-b4aa-2ed7733a8b0e-kube-api-access-cwcwq\") pod \"kube-auth-proxy-5bbd68dc59-6grwd\" (UID: \"415ce0db-109b-4944-b4aa-2ed7733a8b0e\") " pod="openshift-ingress/kube-auth-proxy-5bbd68dc59-6grwd"
Sep 18 06:32:22.214716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:22.214577    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/415ce0db-109b-4944-b4aa-2ed7733a8b0e-tmp\") pod \"kube-auth-proxy-5bbd68dc59-6grwd\" (UID: \"415ce0db-109b-4944-b4aa-2ed7733a8b0e\") " pod="openshift-ingress/kube-auth-proxy-5bbd68dc59-6grwd"
Sep 18 06:32:22.214716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:22.214624    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/415ce0db-109b-4944-b4aa-2ed7733a8b0e-tls-certs\") pod \"kube-auth-proxy-5bbd68dc59-6grwd\" (UID: \"415ce0db-109b-4944-b4aa-2ed7733a8b0e\") " pod="openshift-ingress/kube-auth-proxy-5bbd68dc59-6grwd"
Sep 18 06:32:22.216815 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:22.216792    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/415ce0db-109b-4944-b4aa-2ed7733a8b0e-tmp\") pod \"kube-auth-proxy-5bbd68dc59-6grwd\" (UID: \"415ce0db-109b-4944-b4aa-2ed7733a8b0e\") " pod="openshift-ingress/kube-auth-proxy-5bbd68dc59-6grwd"
Sep 18 06:32:22.217079 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:22.217061    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/415ce0db-109b-4944-b4aa-2ed7733a8b0e-tls-certs\") pod \"kube-auth-proxy-5bbd68dc59-6grwd\" (UID: \"415ce0db-109b-4944-b4aa-2ed7733a8b0e\") " pod="openshift-ingress/kube-auth-proxy-5bbd68dc59-6grwd"
Sep 18 06:32:22.229275 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:22.229249    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwcwq\" (UniqueName: \"kubernetes.io/projected/415ce0db-109b-4944-b4aa-2ed7733a8b0e-kube-api-access-cwcwq\") pod \"kube-auth-proxy-5bbd68dc59-6grwd\" (UID: \"415ce0db-109b-4944-b4aa-2ed7733a8b0e\") " pod="openshift-ingress/kube-auth-proxy-5bbd68dc59-6grwd"
Sep 18 06:32:22.351378 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:22.351302    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5bbd68dc59-6grwd"
Sep 18 06:32:22.672621 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:22.672597    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5bbd68dc59-6grwd"]
Sep 18 06:32:22.674177 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:32:22.674149    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod415ce0db_109b_4944_b4aa_2ed7733a8b0e.slice/crio-cd9159988552d81597750cdfbcf17eadeb8169b9a5d3ff90ff9567a133929aff WatchSource:0}: Error finding container cd9159988552d81597750cdfbcf17eadeb8169b9a5d3ff90ff9567a133929aff: Status 404 returned error can't find the container with id cd9159988552d81597750cdfbcf17eadeb8169b9a5d3ff90ff9567a133929aff
Sep 18 06:32:22.803474 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:22.803442    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5bbd68dc59-6grwd" event={"ID":"415ce0db-109b-4944-b4aa-2ed7733a8b0e","Type":"ContainerStarted","Data":"cd9159988552d81597750cdfbcf17eadeb8169b9a5d3ff90ff9567a133929aff"}
Sep 18 06:32:23.200384 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.200338    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"]
Sep 18 06:32:23.206727 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.205589    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:23.256326 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.256278    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"]
Sep 18 06:32:23.256490 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.256359    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.259085 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.259008    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 18 06:32:23.259085 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.259073    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 18 06:32:23.259313 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.259241    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 18 06:32:23.259386 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.259342    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 18 06:32:23.259386 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.259378    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jrfxp\""
Sep 18 06:32:23.263500 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.263478    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:23.323653 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.323618    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ac21d5d-26a3-4607-847a-50c03f33611b-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-v57nj\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.323653 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.323654    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-v57nj\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.323853 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.323678    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1ac21d5d-26a3-4607-847a-50c03f33611b-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-v57nj\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.323853 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.323733    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-v57nj\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.323853 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.323807    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rttgp\" (UniqueName: \"kubernetes.io/projected/1ac21d5d-26a3-4607-847a-50c03f33611b-kube-api-access-rttgp\") pod \"istiod-openshift-gateway-7b7995d5cd-v57nj\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.323961 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.323872    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-v57nj\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.323961 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.323934    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-v57nj\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.424954 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.424908    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rttgp\" (UniqueName: \"kubernetes.io/projected/1ac21d5d-26a3-4607-847a-50c03f33611b-kube-api-access-rttgp\") pod \"istiod-openshift-gateway-7b7995d5cd-v57nj\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.425135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.424977    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-v57nj\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.425135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.425034    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-v57nj\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.425135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.425066    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ac21d5d-26a3-4607-847a-50c03f33611b-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-v57nj\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.425135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.425093    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-v57nj\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.425135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.425115    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1ac21d5d-26a3-4607-847a-50c03f33611b-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-v57nj\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.425454 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.425138    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-v57nj\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.425966 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.425915    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-v57nj\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.428180 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.428153    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-v57nj\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.428801 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.428757    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-v57nj\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.428903 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.428797    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ac21d5d-26a3-4607-847a-50c03f33611b-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-v57nj\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.429997 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.429969    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1ac21d5d-26a3-4607-847a-50c03f33611b-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-v57nj\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.433243 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.433190    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-v57nj\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.433966 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.433927    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rttgp\" (UniqueName: \"kubernetes.io/projected/1ac21d5d-26a3-4607-847a-50c03f33611b-kube-api-access-rttgp\") pod \"istiod-openshift-gateway-7b7995d5cd-v57nj\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.565690 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.565606    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:23.879560 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:23.879470    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"]
Sep 18 06:32:23.881977 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:32:23.881943    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ac21d5d_26a3_4607_847a_50c03f33611b.slice/crio-2a97702c2bd3bddbaff91087e37d4a62b77251e2454a26245000d85aca2ca3a7 WatchSource:0}: Error finding container 2a97702c2bd3bddbaff91087e37d4a62b77251e2454a26245000d85aca2ca3a7: Status 404 returned error can't find the container with id 2a97702c2bd3bddbaff91087e37d4a62b77251e2454a26245000d85aca2ca3a7
Sep 18 06:32:24.073040 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:24.073001    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:24.817081 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:24.817033    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj" event={"ID":"1ac21d5d-26a3-4607-847a-50c03f33611b","Type":"ContainerStarted","Data":"2a97702c2bd3bddbaff91087e37d4a62b77251e2454a26245000d85aca2ca3a7"}
Sep 18 06:32:25.824452 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:25.824160    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5bbd68dc59-6grwd" event={"ID":"415ce0db-109b-4944-b4aa-2ed7733a8b0e","Type":"ContainerStarted","Data":"2dc5c6ca891660c58097d8061cc59a65c255d464f7b2f77bcbf7f4851a7c63d6"}
Sep 18 06:32:25.840777 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:25.840718    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5bbd68dc59-6grwd" podStartSLOduration=0.861252082 podStartE2EDuration="3.840698155s" totalImagesPullingTime="2.979446073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:32:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:32:25.83895273 +0000 UTC m=+611.520480503" watchObservedRunningTime="2026-09-18 06:32:25.840698155 +0000 UTC m=+611.522225903"
Sep 18 06:32:27.403523 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:27.403484    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:32:27.403795 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:27.403553    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:32:27.835370 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:27.835278    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj" event={"ID":"1ac21d5d-26a3-4607-847a-50c03f33611b","Type":"ContainerStarted","Data":"cf37e317e32d5657ccef093a9c4ae30929e132e329f395831e235fb0ce69e1e7"}
Sep 18 06:32:27.835370 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:27.835349    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:27.855480 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:27.855433    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj" podStartSLOduration=1.336408756 podStartE2EDuration="4.855415203s" totalImagesPullingTime="3.519006447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:32:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:32:27.85300531 +0000 UTC m=+613.534533059" watchObservedRunningTime="2026-09-18 06:32:27.855415203 +0000 UTC m=+613.536942947"
Sep 18 06:32:28.135849 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:28.135758    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:28.195758 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:28.195726    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:28.439482 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:28.439447    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:28.546716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:28.546677    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:28.647285 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:28.647253    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:28.841328 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:28.841255    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:32.636758 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:32.636725    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:32.736295 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:32.736265    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:32.837467 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:32.837437    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:32.937663 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:32.937635    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:33.145661 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:33.145628    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:33.150285 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:33.150254    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nfc2b"]
Sep 18 06:32:33.153350 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:33.153334    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-nfc2b"
Sep 18 06:32:33.156692 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:33.156671    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 18 06:32:33.156793 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:33.156671    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-j26pm\""
Sep 18 06:32:33.163292 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:33.163268    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nfc2b"]
Sep 18 06:32:33.246455 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:33.246374    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:33.321166 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:33.321137    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrgfc\" (UniqueName: \"kubernetes.io/projected/8947c3c0-f29d-44a8-a085-6877f589af98-kube-api-access-hrgfc\") pod \"model-serving-api-54d89bd5fc-nfc2b\" (UID: \"8947c3c0-f29d-44a8-a085-6877f589af98\") " pod="opendatahub/model-serving-api-54d89bd5fc-nfc2b"
Sep 18 06:32:33.321339 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:33.321196    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8947c3c0-f29d-44a8-a085-6877f589af98-tls-certs\") pod \"model-serving-api-54d89bd5fc-nfc2b\" (UID: \"8947c3c0-f29d-44a8-a085-6877f589af98\") " pod="opendatahub/model-serving-api-54d89bd5fc-nfc2b"
Sep 18 06:32:33.422034 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:33.422001    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrgfc\" (UniqueName: \"kubernetes.io/projected/8947c3c0-f29d-44a8-a085-6877f589af98-kube-api-access-hrgfc\") pod \"model-serving-api-54d89bd5fc-nfc2b\" (UID: \"8947c3c0-f29d-44a8-a085-6877f589af98\") " pod="opendatahub/model-serving-api-54d89bd5fc-nfc2b"
Sep 18 06:32:33.422181 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:33.422085    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8947c3c0-f29d-44a8-a085-6877f589af98-tls-certs\") pod \"model-serving-api-54d89bd5fc-nfc2b\" (UID: \"8947c3c0-f29d-44a8-a085-6877f589af98\") " pod="opendatahub/model-serving-api-54d89bd5fc-nfc2b"
Sep 18 06:32:33.424436 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:33.424413    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8947c3c0-f29d-44a8-a085-6877f589af98-tls-certs\") pod \"model-serving-api-54d89bd5fc-nfc2b\" (UID: \"8947c3c0-f29d-44a8-a085-6877f589af98\") " pod="opendatahub/model-serving-api-54d89bd5fc-nfc2b"
Sep 18 06:32:33.429050 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:33.429026    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrgfc\" (UniqueName: \"kubernetes.io/projected/8947c3c0-f29d-44a8-a085-6877f589af98-kube-api-access-hrgfc\") pod \"model-serving-api-54d89bd5fc-nfc2b\" (UID: \"8947c3c0-f29d-44a8-a085-6877f589af98\") " pod="opendatahub/model-serving-api-54d89bd5fc-nfc2b"
Sep 18 06:32:33.464179 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:33.464154    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-nfc2b"
Sep 18 06:32:33.730911 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:33.730880    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nfc2b"]
Sep 18 06:32:33.734180 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:32:33.734154    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8947c3c0_f29d_44a8_a085_6877f589af98.slice/crio-6cf8378dc5b4d7ef36ec5e56772228a81b8e1c4aeabe79367b3aa826ef657de2 WatchSource:0}: Error finding container 6cf8378dc5b4d7ef36ec5e56772228a81b8e1c4aeabe79367b3aa826ef657de2: Status 404 returned error can't find the container with id 6cf8378dc5b4d7ef36ec5e56772228a81b8e1c4aeabe79367b3aa826ef657de2
Sep 18 06:32:33.736476 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:33.736458    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 06:32:33.860983 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:33.860907    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-nfc2b" event={"ID":"8947c3c0-f29d-44a8-a085-6877f589af98","Type":"ContainerStarted","Data":"6cf8378dc5b4d7ef36ec5e56772228a81b8e1c4aeabe79367b3aa826ef657de2"}
Sep 18 06:32:34.606550 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:34.606519    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:35.870174 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:35.870090    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-nfc2b" event={"ID":"8947c3c0-f29d-44a8-a085-6877f589af98","Type":"ContainerStarted","Data":"f0b8b3e86210cdf5f4cb6885ec370321f0d76240e4cf7399bd09c916d034dcd6"}
Sep 18 06:32:35.870547 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:35.870238    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-nfc2b"
Sep 18 06:32:35.886182 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:35.886137    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-nfc2b" podStartSLOduration=1.024079396 podStartE2EDuration="2.886121402s" totalImagesPullingTime="1.862042006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:32:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:32:35.884829897 +0000 UTC m=+621.566357641" watchObservedRunningTime="2026-09-18 06:32:35.886121402 +0000 UTC m=+621.567649147"
Sep 18 06:32:36.887760 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:36.887729    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:36.986892 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:36.986835    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:39.803053 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:39.803013    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-797df4568f-g8jcs"]
Sep 18 06:32:39.806438 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:39.806417    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:39.818459 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:39.818435    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-797df4568f-g8jcs"]
Sep 18 06:32:39.862587 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:39.862544    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:39.981417 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:39.981385    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/042d2099-832e-4b63-9da0-88f08a90e8ce-console-serving-cert\") pod \"console-797df4568f-g8jcs\" (UID: \"042d2099-832e-4b63-9da0-88f08a90e8ce\") " pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:39.981613 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:39.981427    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/042d2099-832e-4b63-9da0-88f08a90e8ce-console-oauth-config\") pod \"console-797df4568f-g8jcs\" (UID: \"042d2099-832e-4b63-9da0-88f08a90e8ce\") " pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:39.981613 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:39.981450    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/042d2099-832e-4b63-9da0-88f08a90e8ce-trusted-ca-bundle\") pod \"console-797df4568f-g8jcs\" (UID: \"042d2099-832e-4b63-9da0-88f08a90e8ce\") " pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:39.981613 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:39.981532    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/042d2099-832e-4b63-9da0-88f08a90e8ce-service-ca\") pod \"console-797df4568f-g8jcs\" (UID: \"042d2099-832e-4b63-9da0-88f08a90e8ce\") " pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:39.981613 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:39.981572    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/042d2099-832e-4b63-9da0-88f08a90e8ce-console-config\") pod \"console-797df4568f-g8jcs\" (UID: \"042d2099-832e-4b63-9da0-88f08a90e8ce\") " pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:39.981788 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:39.981656    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/042d2099-832e-4b63-9da0-88f08a90e8ce-oauth-serving-cert\") pod \"console-797df4568f-g8jcs\" (UID: \"042d2099-832e-4b63-9da0-88f08a90e8ce\") " pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:39.981788 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:39.981690    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzzdk\" (UniqueName: \"kubernetes.io/projected/042d2099-832e-4b63-9da0-88f08a90e8ce-kube-api-access-kzzdk\") pod \"console-797df4568f-g8jcs\" (UID: \"042d2099-832e-4b63-9da0-88f08a90e8ce\") " pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:40.007522 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.007483    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:40.082874 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.082789    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/042d2099-832e-4b63-9da0-88f08a90e8ce-service-ca\") pod \"console-797df4568f-g8jcs\" (UID: \"042d2099-832e-4b63-9da0-88f08a90e8ce\") " pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:40.082874 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.082824    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/042d2099-832e-4b63-9da0-88f08a90e8ce-console-config\") pod \"console-797df4568f-g8jcs\" (UID: \"042d2099-832e-4b63-9da0-88f08a90e8ce\") " pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:40.082874 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.082862    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/042d2099-832e-4b63-9da0-88f08a90e8ce-oauth-serving-cert\") pod \"console-797df4568f-g8jcs\" (UID: \"042d2099-832e-4b63-9da0-88f08a90e8ce\") " pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:40.083134 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.082883    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzzdk\" (UniqueName: \"kubernetes.io/projected/042d2099-832e-4b63-9da0-88f08a90e8ce-kube-api-access-kzzdk\") pod \"console-797df4568f-g8jcs\" (UID: \"042d2099-832e-4b63-9da0-88f08a90e8ce\") " pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:40.083134 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.082919    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/042d2099-832e-4b63-9da0-88f08a90e8ce-console-serving-cert\") pod \"console-797df4568f-g8jcs\" (UID: \"042d2099-832e-4b63-9da0-88f08a90e8ce\") " pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:40.083134 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.082945    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/042d2099-832e-4b63-9da0-88f08a90e8ce-console-oauth-config\") pod \"console-797df4568f-g8jcs\" (UID: \"042d2099-832e-4b63-9da0-88f08a90e8ce\") " pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:40.083134 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.082972    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/042d2099-832e-4b63-9da0-88f08a90e8ce-trusted-ca-bundle\") pod \"console-797df4568f-g8jcs\" (UID: \"042d2099-832e-4b63-9da0-88f08a90e8ce\") " pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:40.083723 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.083695    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/042d2099-832e-4b63-9da0-88f08a90e8ce-service-ca\") pod \"console-797df4568f-g8jcs\" (UID: \"042d2099-832e-4b63-9da0-88f08a90e8ce\") " pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:40.083723 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.083717    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/042d2099-832e-4b63-9da0-88f08a90e8ce-console-config\") pod \"console-797df4568f-g8jcs\" (UID: \"042d2099-832e-4b63-9da0-88f08a90e8ce\") " pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:40.083894 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.083698    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/042d2099-832e-4b63-9da0-88f08a90e8ce-oauth-serving-cert\") pod \"console-797df4568f-g8jcs\" (UID: \"042d2099-832e-4b63-9da0-88f08a90e8ce\") " pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:40.084029 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.084007    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/042d2099-832e-4b63-9da0-88f08a90e8ce-trusted-ca-bundle\") pod \"console-797df4568f-g8jcs\" (UID: \"042d2099-832e-4b63-9da0-88f08a90e8ce\") " pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:40.085609 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.085583    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/042d2099-832e-4b63-9da0-88f08a90e8ce-console-oauth-config\") pod \"console-797df4568f-g8jcs\" (UID: \"042d2099-832e-4b63-9da0-88f08a90e8ce\") " pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:40.085718 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.085636    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/042d2099-832e-4b63-9da0-88f08a90e8ce-console-serving-cert\") pod \"console-797df4568f-g8jcs\" (UID: \"042d2099-832e-4b63-9da0-88f08a90e8ce\") " pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:40.094427 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.094407    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzzdk\" (UniqueName: \"kubernetes.io/projected/042d2099-832e-4b63-9da0-88f08a90e8ce-kube-api-access-kzzdk\") pod \"console-797df4568f-g8jcs\" (UID: \"042d2099-832e-4b63-9da0-88f08a90e8ce\") " pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:40.107453 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.107424    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:40.115393 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.115378    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:40.407990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.407965    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-797df4568f-g8jcs"]
Sep 18 06:32:40.409659 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:32:40.409635    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod042d2099_832e_4b63_9da0_88f08a90e8ce.slice/crio-742d55fc06f4262dc5a76dc04d5cd6f15d4958865dac7c15c6d09094de94234d WatchSource:0}: Error finding container 742d55fc06f4262dc5a76dc04d5cd6f15d4958865dac7c15c6d09094de94234d: Status 404 returned error can't find the container with id 742d55fc06f4262dc5a76dc04d5cd6f15d4958865dac7c15c6d09094de94234d
Sep 18 06:32:40.833480 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.833349    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:40.891186 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.891152    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797df4568f-g8jcs" event={"ID":"042d2099-832e-4b63-9da0-88f08a90e8ce","Type":"ContainerStarted","Data":"fc09c9c013b9471c013c8bbaeec302e01c5bbf57fd515bac15a538793671f4f4"}
Sep 18 06:32:40.891384 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.891201    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-797df4568f-g8jcs" event={"ID":"042d2099-832e-4b63-9da0-88f08a90e8ce","Type":"ContainerStarted","Data":"742d55fc06f4262dc5a76dc04d5cd6f15d4958865dac7c15c6d09094de94234d"}
Sep 18 06:32:40.910872 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.910826    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-797df4568f-g8jcs" podStartSLOduration=1.9108109340000001 podStartE2EDuration="1.910810934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:32:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:32:40.909780941 +0000 UTC m=+626.591308686" watchObservedRunningTime="2026-09-18 06:32:40.910810934 +0000 UTC m=+626.592338678"
Sep 18 06:32:40.914177 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:40.914154    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:42.156087 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.156053    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:42.211299 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.211266    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jz5zd"]
Sep 18 06:32:42.214668 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.214647    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jz5zd"
Sep 18 06:32:42.217504 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.217470    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 18 06:32:42.217628 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.217508    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-4fx87\""
Sep 18 06:32:42.227607 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.227584    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jz5zd"]
Sep 18 06:32:42.230789 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.230765    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5dl2q"]
Sep 18 06:32:42.234407 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.234390    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5dl2q"
Sep 18 06:32:42.236919 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.236899    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 18 06:32:42.237006 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.236964    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-8x22b\""
Sep 18 06:32:42.243412 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.243387    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5dl2q"]
Sep 18 06:32:42.244234 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.244179    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:42.300156 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.300118    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24e331ac-1ca6-484e-a96a-2603f3f51732-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5dl2q\" (UID: \"24e331ac-1ca6-484e-a96a-2603f3f51732\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5dl2q"
Sep 18 06:32:42.300328 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.300252    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrxg9\" (UniqueName: \"kubernetes.io/projected/d377a298-99f3-4042-98c8-303150a24782-kube-api-access-hrxg9\") pod \"kserve-controller-manager-d6f4b65b8-jz5zd\" (UID: \"d377a298-99f3-4042-98c8-303150a24782\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jz5zd"
Sep 18 06:32:42.300328 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.300315    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d377a298-99f3-4042-98c8-303150a24782-cert\") pod \"kserve-controller-manager-d6f4b65b8-jz5zd\" (UID: \"d377a298-99f3-4042-98c8-303150a24782\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jz5zd"
Sep 18 06:32:42.300474 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.300366    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zn8b\" (UniqueName: \"kubernetes.io/projected/24e331ac-1ca6-484e-a96a-2603f3f51732-kube-api-access-9zn8b\") pod \"llmisvc-controller-manager-777c6bd7d5-5dl2q\" (UID: \"24e331ac-1ca6-484e-a96a-2603f3f51732\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5dl2q"
Sep 18 06:32:42.320289 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.320258    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:42.393103 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.393071    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:42.401352 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.401329    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24e331ac-1ca6-484e-a96a-2603f3f51732-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5dl2q\" (UID: \"24e331ac-1ca6-484e-a96a-2603f3f51732\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5dl2q"
Sep 18 06:32:42.401415 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.401379    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrxg9\" (UniqueName: \"kubernetes.io/projected/d377a298-99f3-4042-98c8-303150a24782-kube-api-access-hrxg9\") pod \"kserve-controller-manager-d6f4b65b8-jz5zd\" (UID: \"d377a298-99f3-4042-98c8-303150a24782\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jz5zd"
Sep 18 06:32:42.401415 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.401409    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d377a298-99f3-4042-98c8-303150a24782-cert\") pod \"kserve-controller-manager-d6f4b65b8-jz5zd\" (UID: \"d377a298-99f3-4042-98c8-303150a24782\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jz5zd"
Sep 18 06:32:42.401490 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.401440    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zn8b\" (UniqueName: \"kubernetes.io/projected/24e331ac-1ca6-484e-a96a-2603f3f51732-kube-api-access-9zn8b\") pod \"llmisvc-controller-manager-777c6bd7d5-5dl2q\" (UID: \"24e331ac-1ca6-484e-a96a-2603f3f51732\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5dl2q"
Sep 18 06:32:42.403830 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.403800    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d377a298-99f3-4042-98c8-303150a24782-cert\") pod \"kserve-controller-manager-d6f4b65b8-jz5zd\" (UID: \"d377a298-99f3-4042-98c8-303150a24782\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jz5zd"
Sep 18 06:32:42.403953 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.403935    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/24e331ac-1ca6-484e-a96a-2603f3f51732-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-5dl2q\" (UID: \"24e331ac-1ca6-484e-a96a-2603f3f51732\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5dl2q"
Sep 18 06:32:42.415179 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.415128    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zn8b\" (UniqueName: \"kubernetes.io/projected/24e331ac-1ca6-484e-a96a-2603f3f51732-kube-api-access-9zn8b\") pod \"llmisvc-controller-manager-777c6bd7d5-5dl2q\" (UID: \"24e331ac-1ca6-484e-a96a-2603f3f51732\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5dl2q"
Sep 18 06:32:42.415283 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.415213    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrxg9\" (UniqueName: \"kubernetes.io/projected/d377a298-99f3-4042-98c8-303150a24782-kube-api-access-hrxg9\") pod \"kserve-controller-manager-d6f4b65b8-jz5zd\" (UID: \"d377a298-99f3-4042-98c8-303150a24782\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jz5zd"
Sep 18 06:32:42.528628 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.528586    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jz5zd"
Sep 18 06:32:42.545956 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.545933    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5dl2q"
Sep 18 06:32:42.798938 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.798916    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jz5zd"]
Sep 18 06:32:42.799982 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:32:42.799958    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd377a298_99f3_4042_98c8_303150a24782.slice/crio-f8377cd0a730ea3765bce9d818b89ae663a5be9cc81e498c3402f951fe76b58b WatchSource:0}: Error finding container f8377cd0a730ea3765bce9d818b89ae663a5be9cc81e498c3402f951fe76b58b: Status 404 returned error can't find the container with id f8377cd0a730ea3765bce9d818b89ae663a5be9cc81e498c3402f951fe76b58b
Sep 18 06:32:42.820472 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.820453    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-5dl2q"]
Sep 18 06:32:42.822590 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:32:42.822563    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod24e331ac_1ca6_484e_a96a_2603f3f51732.slice/crio-ddeb5d6b3c3adcb7fb1cc2234430e14daed7a10887d2f3864a9ee3c519474e21 WatchSource:0}: Error finding container ddeb5d6b3c3adcb7fb1cc2234430e14daed7a10887d2f3864a9ee3c519474e21: Status 404 returned error can't find the container with id ddeb5d6b3c3adcb7fb1cc2234430e14daed7a10887d2f3864a9ee3c519474e21
Sep 18 06:32:42.901038 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.901009    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5dl2q" event={"ID":"24e331ac-1ca6-484e-a96a-2603f3f51732","Type":"ContainerStarted","Data":"ddeb5d6b3c3adcb7fb1cc2234430e14daed7a10887d2f3864a9ee3c519474e21"}
Sep 18 06:32:42.902115 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:42.902094    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jz5zd" event={"ID":"d377a298-99f3-4042-98c8-303150a24782","Type":"ContainerStarted","Data":"f8377cd0a730ea3765bce9d818b89ae663a5be9cc81e498c3402f951fe76b58b"}
Sep 18 06:32:43.418565 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:43.418537    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:43.535281 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:43.535241    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:43.642515 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:43.642475    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:46.879309 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:46.879278    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-nfc2b"
Sep 18 06:32:46.884087 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:46.884063    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"]
Sep 18 06:32:46.884263 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:46.884241    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj" podUID="1ac21d5d-26a3-4607-847a-50c03f33611b" containerName="discovery" containerID="cri-o://cf37e317e32d5657ccef093a9c4ae30929e132e329f395831e235fb0ce69e1e7" gracePeriod=30
Sep 18 06:32:46.929785 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:46.929753    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jz5zd" event={"ID":"d377a298-99f3-4042-98c8-303150a24782","Type":"ContainerStarted","Data":"59aad30888647bdcefc0ff08f54beaf93dfd97b91fdd0b14f94b872f500df347"}
Sep 18 06:32:46.929923 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:46.929822    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jz5zd"
Sep 18 06:32:46.931636 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:46.931610    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5dl2q" event={"ID":"24e331ac-1ca6-484e-a96a-2603f3f51732","Type":"ContainerStarted","Data":"c567117529e7fa8f053dda507d9a6d2a57b045a3eb893867422f11d5f7b988a8"}
Sep 18 06:32:46.931755 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:46.931670    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5dl2q"
Sep 18 06:32:46.945899 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:46.945856    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jz5zd" podStartSLOduration=1.886325391 podStartE2EDuration="4.945841566s" totalImagesPullingTime="3.059516175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:32:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:32:46.944959488 +0000 UTC m=+632.626487233" watchObservedRunningTime="2026-09-18 06:32:46.945841566 +0000 UTC m=+632.627369310"
Sep 18 06:32:46.960846 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:46.960805    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5dl2q" podStartSLOduration=1.924001759 podStartE2EDuration="4.96078952s" totalImagesPullingTime="3.036787761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:32:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:32:46.959940248 +0000 UTC m=+632.641467989" watchObservedRunningTime="2026-09-18 06:32:46.96078952 +0000 UTC m=+632.642317310"
Sep 18 06:32:47.136841 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.136786    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:47.244398 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.244367    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ac21d5d-26a3-4607-847a-50c03f33611b-local-certs\") pod \"1ac21d5d-26a3-4607-847a-50c03f33611b\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") "
Sep 18 06:32:47.244398 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.244398    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1ac21d5d-26a3-4607-847a-50c03f33611b-cacerts\") pod \"1ac21d5d-26a3-4607-847a-50c03f33611b\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") "
Sep 18 06:32:47.244638 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.244464    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-kubeconfig\") pod \"1ac21d5d-26a3-4607-847a-50c03f33611b\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") "
Sep 18 06:32:47.244638 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.244485    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-csr-ca-configmap\") pod \"1ac21d5d-26a3-4607-847a-50c03f33611b\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") "
Sep 18 06:32:47.244638 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.244501    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-token\") pod \"1ac21d5d-26a3-4607-847a-50c03f33611b\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") "
Sep 18 06:32:47.244638 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.244551    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rttgp\" (UniqueName: \"kubernetes.io/projected/1ac21d5d-26a3-4607-847a-50c03f33611b-kube-api-access-rttgp\") pod \"1ac21d5d-26a3-4607-847a-50c03f33611b\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") "
Sep 18 06:32:47.244638 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.244595    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-csr-dns-cert\") pod \"1ac21d5d-26a3-4607-847a-50c03f33611b\" (UID: \"1ac21d5d-26a3-4607-847a-50c03f33611b\") "
Sep 18 06:32:47.244934 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.244905    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "1ac21d5d-26a3-4607-847a-50c03f33611b" (UID: "1ac21d5d-26a3-4607-847a-50c03f33611b"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 06:32:47.246942 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.246911    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "1ac21d5d-26a3-4607-847a-50c03f33611b" (UID: "1ac21d5d-26a3-4607-847a-50c03f33611b"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:32:47.246942 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.246925    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ac21d5d-26a3-4607-847a-50c03f33611b-cacerts" (OuterVolumeSpecName: "cacerts") pod "1ac21d5d-26a3-4607-847a-50c03f33611b" (UID: "1ac21d5d-26a3-4607-847a-50c03f33611b"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:32:47.247131 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.247030    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-token" (OuterVolumeSpecName: "istio-token") pod "1ac21d5d-26a3-4607-847a-50c03f33611b" (UID: "1ac21d5d-26a3-4607-847a-50c03f33611b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:32:47.247131 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.247089    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "1ac21d5d-26a3-4607-847a-50c03f33611b" (UID: "1ac21d5d-26a3-4607-847a-50c03f33611b"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:32:47.247131 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.247112    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ac21d5d-26a3-4607-847a-50c03f33611b-kube-api-access-rttgp" (OuterVolumeSpecName: "kube-api-access-rttgp") pod "1ac21d5d-26a3-4607-847a-50c03f33611b" (UID: "1ac21d5d-26a3-4607-847a-50c03f33611b"). InnerVolumeSpecName "kube-api-access-rttgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:32:47.247411 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.247386    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ac21d5d-26a3-4607-847a-50c03f33611b-local-certs" (OuterVolumeSpecName: "local-certs") pod "1ac21d5d-26a3-4607-847a-50c03f33611b" (UID: "1ac21d5d-26a3-4607-847a-50c03f33611b"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:32:47.345283 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.345250    2568 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-kubeconfig\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:32:47.345283 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.345282    2568 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-csr-ca-configmap\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:32:47.345283 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.345297    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-token\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:32:47.345551 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.345310    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rttgp\" (UniqueName: \"kubernetes.io/projected/1ac21d5d-26a3-4607-847a-50c03f33611b-kube-api-access-rttgp\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:32:47.345551 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.345320    2568 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1ac21d5d-26a3-4607-847a-50c03f33611b-istio-csr-dns-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:32:47.345551 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.345331    2568 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ac21d5d-26a3-4607-847a-50c03f33611b-local-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:32:47.345551 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.345342    2568 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1ac21d5d-26a3-4607-847a-50c03f33611b-cacerts\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:32:47.937548 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.937512    2568 generic.go:356] "Generic (PLEG): container finished" podID="1ac21d5d-26a3-4607-847a-50c03f33611b" containerID="cf37e317e32d5657ccef093a9c4ae30929e132e329f395831e235fb0ce69e1e7" exitCode=0
Sep 18 06:32:47.937953 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.937560    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"
Sep 18 06:32:47.937953 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.937597    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj" event={"ID":"1ac21d5d-26a3-4607-847a-50c03f33611b","Type":"ContainerDied","Data":"cf37e317e32d5657ccef093a9c4ae30929e132e329f395831e235fb0ce69e1e7"}
Sep 18 06:32:47.937953 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.937657    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj" event={"ID":"1ac21d5d-26a3-4607-847a-50c03f33611b","Type":"ContainerDied","Data":"2a97702c2bd3bddbaff91087e37d4a62b77251e2454a26245000d85aca2ca3a7"}
Sep 18 06:32:47.937953 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.937680    2568 scope.go:117] "RemoveContainer" containerID="cf37e317e32d5657ccef093a9c4ae30929e132e329f395831e235fb0ce69e1e7"
Sep 18 06:32:47.946707 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.946690    2568 scope.go:117] "RemoveContainer" containerID="cf37e317e32d5657ccef093a9c4ae30929e132e329f395831e235fb0ce69e1e7"
Sep 18 06:32:47.946977 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:32:47.946956    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf37e317e32d5657ccef093a9c4ae30929e132e329f395831e235fb0ce69e1e7\": container with ID starting with cf37e317e32d5657ccef093a9c4ae30929e132e329f395831e235fb0ce69e1e7 not found: ID does not exist" containerID="cf37e317e32d5657ccef093a9c4ae30929e132e329f395831e235fb0ce69e1e7"
Sep 18 06:32:47.947058 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.946984    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf37e317e32d5657ccef093a9c4ae30929e132e329f395831e235fb0ce69e1e7"} err="failed to get container status \"cf37e317e32d5657ccef093a9c4ae30929e132e329f395831e235fb0ce69e1e7\": rpc error: code = NotFound desc = could not find container \"cf37e317e32d5657ccef093a9c4ae30929e132e329f395831e235fb0ce69e1e7\": container with ID starting with cf37e317e32d5657ccef093a9c4ae30929e132e329f395831e235fb0ce69e1e7 not found: ID does not exist"
Sep 18 06:32:47.958374 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.958351    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"]
Sep 18 06:32:47.961577 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:47.961557    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj"]
Sep 18 06:32:48.547389 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:48.547355    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ac21d5d-26a3-4607-847a-50c03f33611b" path="/var/lib/kubelet/pods/1ac21d5d-26a3-4607-847a-50c03f33611b/volumes"
Sep 18 06:32:50.116131 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:50.116095    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:50.116553 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:50.116149    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:50.120830 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:50.120807    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:50.955575 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:50.955549    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-797df4568f-g8jcs"
Sep 18 06:32:50.997815 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:50.997789    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b87fd654d-v9swj"]
Sep 18 06:32:51.095836 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:51.095795    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:51.196794 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:51.196763    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:32:51.256955 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:32:51.256885    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:33:16.016755 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.016709    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5b87fd654d-v9swj" podUID="9907de51-d4f8-4067-ad96-1a1e5e3448ad" containerName="console" containerID="cri-o://84b5c06db093703b3f13b12fd24abe908f85573538fc4d422f6b3e76af220491" gracePeriod=15
Sep 18 06:33:16.267029 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.266972    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:33:16.401720 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.401688    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-oauth-serving-cert\") pod \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") "
Sep 18 06:33:16.401892 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.401735    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-service-ca\") pod \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") "
Sep 18 06:33:16.401892 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.401768    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9907de51-d4f8-4067-ad96-1a1e5e3448ad-console-oauth-config\") pod \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") "
Sep 18 06:33:16.401892 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.401825    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-trusted-ca-bundle\") pod \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") "
Sep 18 06:33:16.402059 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.401942    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-console-config\") pod \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") "
Sep 18 06:33:16.402059 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.401985    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lm5h\" (UniqueName: \"kubernetes.io/projected/9907de51-d4f8-4067-ad96-1a1e5e3448ad-kube-api-access-7lm5h\") pod \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") "
Sep 18 06:33:16.402059 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.402020    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9907de51-d4f8-4067-ad96-1a1e5e3448ad-console-serving-cert\") pod \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\" (UID: \"9907de51-d4f8-4067-ad96-1a1e5e3448ad\") "
Sep 18 06:33:16.402207 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.402100    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9907de51-d4f8-4067-ad96-1a1e5e3448ad" (UID: "9907de51-d4f8-4067-ad96-1a1e5e3448ad"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 06:33:16.402292 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.402258    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-service-ca" (OuterVolumeSpecName: "service-ca") pod "9907de51-d4f8-4067-ad96-1a1e5e3448ad" (UID: "9907de51-d4f8-4067-ad96-1a1e5e3448ad"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 06:33:16.402292 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.402270    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9907de51-d4f8-4067-ad96-1a1e5e3448ad" (UID: "9907de51-d4f8-4067-ad96-1a1e5e3448ad"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 06:33:16.402388 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.402326    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-console-config" (OuterVolumeSpecName: "console-config") pod "9907de51-d4f8-4067-ad96-1a1e5e3448ad" (UID: "9907de51-d4f8-4067-ad96-1a1e5e3448ad"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 06:33:16.402437 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.402404    2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-trusted-ca-bundle\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:33:16.402437 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.402421    2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-console-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:33:16.402437 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.402432    2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-oauth-serving-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:33:16.402437 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.402440    2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9907de51-d4f8-4067-ad96-1a1e5e3448ad-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:33:16.404011 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.403988    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9907de51-d4f8-4067-ad96-1a1e5e3448ad-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9907de51-d4f8-4067-ad96-1a1e5e3448ad" (UID: "9907de51-d4f8-4067-ad96-1a1e5e3448ad"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:33:16.404107 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.404019    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9907de51-d4f8-4067-ad96-1a1e5e3448ad-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9907de51-d4f8-4067-ad96-1a1e5e3448ad" (UID: "9907de51-d4f8-4067-ad96-1a1e5e3448ad"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:33:16.404107 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.404094    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9907de51-d4f8-4067-ad96-1a1e5e3448ad-kube-api-access-7lm5h" (OuterVolumeSpecName: "kube-api-access-7lm5h") pod "9907de51-d4f8-4067-ad96-1a1e5e3448ad" (UID: "9907de51-d4f8-4067-ad96-1a1e5e3448ad"). InnerVolumeSpecName "kube-api-access-7lm5h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:33:16.503814 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.503789    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lm5h\" (UniqueName: \"kubernetes.io/projected/9907de51-d4f8-4067-ad96-1a1e5e3448ad-kube-api-access-7lm5h\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:33:16.503814 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.503812    2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9907de51-d4f8-4067-ad96-1a1e5e3448ad-console-serving-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:33:16.503814 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:16.503822    2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9907de51-d4f8-4067-ad96-1a1e5e3448ad-console-oauth-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:33:17.061126 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:17.061091    2568 generic.go:356] "Generic (PLEG): container finished" podID="9907de51-d4f8-4067-ad96-1a1e5e3448ad" containerID="84b5c06db093703b3f13b12fd24abe908f85573538fc4d422f6b3e76af220491" exitCode=0
Sep 18 06:33:17.061567 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:17.061149    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b87fd654d-v9swj"
Sep 18 06:33:17.061567 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:17.061157    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b87fd654d-v9swj" event={"ID":"9907de51-d4f8-4067-ad96-1a1e5e3448ad","Type":"ContainerDied","Data":"84b5c06db093703b3f13b12fd24abe908f85573538fc4d422f6b3e76af220491"}
Sep 18 06:33:17.061567 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:17.061244    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b87fd654d-v9swj" event={"ID":"9907de51-d4f8-4067-ad96-1a1e5e3448ad","Type":"ContainerDied","Data":"0fe3b79ab9fc8487f06ad1760769fc610d84cd6d73fe155b1bcfb25a4073b9d8"}
Sep 18 06:33:17.061567 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:17.061269    2568 scope.go:117] "RemoveContainer" containerID="84b5c06db093703b3f13b12fd24abe908f85573538fc4d422f6b3e76af220491"
Sep 18 06:33:17.069172 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:17.069153    2568 scope.go:117] "RemoveContainer" containerID="84b5c06db093703b3f13b12fd24abe908f85573538fc4d422f6b3e76af220491"
Sep 18 06:33:17.069454 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:33:17.069434    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84b5c06db093703b3f13b12fd24abe908f85573538fc4d422f6b3e76af220491\": container with ID starting with 84b5c06db093703b3f13b12fd24abe908f85573538fc4d422f6b3e76af220491 not found: ID does not exist" containerID="84b5c06db093703b3f13b12fd24abe908f85573538fc4d422f6b3e76af220491"
Sep 18 06:33:17.069513 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:17.069462    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84b5c06db093703b3f13b12fd24abe908f85573538fc4d422f6b3e76af220491"} err="failed to get container status \"84b5c06db093703b3f13b12fd24abe908f85573538fc4d422f6b3e76af220491\": rpc error: code = NotFound desc = could not find container \"84b5c06db093703b3f13b12fd24abe908f85573538fc4d422f6b3e76af220491\": container with ID starting with 84b5c06db093703b3f13b12fd24abe908f85573538fc4d422f6b3e76af220491 not found: ID does not exist"
Sep 18 06:33:17.077115 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:17.077094    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b87fd654d-v9swj"]
Sep 18 06:33:17.080148 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:17.080128    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b87fd654d-v9swj"]
Sep 18 06:33:17.940071 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:17.940041    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-5dl2q"
Sep 18 06:33:17.943392 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:17.943375    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jz5zd"
Sep 18 06:33:18.543063 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:18.543031    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9907de51-d4f8-4067-ad96-1a1e5e3448ad" path="/var/lib/kubelet/pods/9907de51-d4f8-4067-ad96-1a1e5e3448ad/volumes"
Sep 18 06:33:18.940791 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:18.940759    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:33:33.643730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:33.643697    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:33:33.743186 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:33.743154    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:33:45.878786 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:45.878754    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:33:46.686577 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:46.686545    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:33:49.152189 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:49.152156    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-q78cp"]
Sep 18 06:33:49.152660 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:49.152625    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9907de51-d4f8-4067-ad96-1a1e5e3448ad" containerName="console"
Sep 18 06:33:49.152660 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:49.152644    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9907de51-d4f8-4067-ad96-1a1e5e3448ad" containerName="console"
Sep 18 06:33:49.152783 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:49.152666    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ac21d5d-26a3-4607-847a-50c03f33611b" containerName="discovery"
Sep 18 06:33:49.152783 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:49.152675    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ac21d5d-26a3-4607-847a-50c03f33611b" containerName="discovery"
Sep 18 06:33:49.152783 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:49.152756    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9907de51-d4f8-4067-ad96-1a1e5e3448ad" containerName="console"
Sep 18 06:33:49.152783 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:49.152766    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ac21d5d-26a3-4607-847a-50c03f33611b" containerName="discovery"
Sep 18 06:33:49.155729 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:49.155709    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-q78cp"
Sep 18 06:33:49.159259 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:49.159241    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 18 06:33:49.159367 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:49.159315    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 18 06:33:49.159433 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:49.159320    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-z2frc\""
Sep 18 06:33:49.166388 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:49.166368    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-q78cp"]
Sep 18 06:33:49.279583 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:49.279551    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h945\" (UniqueName: \"kubernetes.io/projected/3c03039b-93f8-489e-8675-9af35729e24b-kube-api-access-8h945\") pod \"authorino-operator-76b478bd78-q78cp\" (UID: \"3c03039b-93f8-489e-8675-9af35729e24b\") " pod="kuadrant-system/authorino-operator-76b478bd78-q78cp"
Sep 18 06:33:49.380082 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:49.380050    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h945\" (UniqueName: \"kubernetes.io/projected/3c03039b-93f8-489e-8675-9af35729e24b-kube-api-access-8h945\") pod \"authorino-operator-76b478bd78-q78cp\" (UID: \"3c03039b-93f8-489e-8675-9af35729e24b\") " pod="kuadrant-system/authorino-operator-76b478bd78-q78cp"
Sep 18 06:33:49.396999 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:49.396967    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h945\" (UniqueName: \"kubernetes.io/projected/3c03039b-93f8-489e-8675-9af35729e24b-kube-api-access-8h945\") pod \"authorino-operator-76b478bd78-q78cp\" (UID: \"3c03039b-93f8-489e-8675-9af35729e24b\") " pod="kuadrant-system/authorino-operator-76b478bd78-q78cp"
Sep 18 06:33:49.465931 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:49.465867    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-q78cp"
Sep 18 06:33:49.735395 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:49.735366    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-q78cp"]
Sep 18 06:33:49.737973 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:33:49.737945    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c03039b_93f8_489e_8675_9af35729e24b.slice/crio-945f4b739cc757e7d48b5e26cd7e62f5e03f5a5002394187a2edd5cfbfd8f589 WatchSource:0}: Error finding container 945f4b739cc757e7d48b5e26cd7e62f5e03f5a5002394187a2edd5cfbfd8f589: Status 404 returned error can't find the container with id 945f4b739cc757e7d48b5e26cd7e62f5e03f5a5002394187a2edd5cfbfd8f589
Sep 18 06:33:49.738589 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:49.738553    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:33:49.890588 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:49.890544    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:33:50.206059 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:50.206023    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-q78cp" event={"ID":"3c03039b-93f8-489e-8675-9af35729e24b","Type":"ContainerStarted","Data":"945f4b739cc757e7d48b5e26cd7e62f5e03f5a5002394187a2edd5cfbfd8f589"}
Sep 18 06:33:51.373694 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:51.373658    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js"]
Sep 18 06:33:51.382237 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:51.382192    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js"
Sep 18 06:33:51.386248 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:51.385963    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-9ms5s\""
Sep 18 06:33:51.387179 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:51.387150    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js"]
Sep 18 06:33:51.500882 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:51.500852    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg6rw\" (UniqueName: \"kubernetes.io/projected/1106342b-2c61-479e-bea5-1004dea87640-kube-api-access-rg6rw\") pod \"limitador-operator-controller-manager-74495c69f8-bk7js\" (UID: \"1106342b-2c61-479e-bea5-1004dea87640\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js"
Sep 18 06:33:51.601487 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:51.601457    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg6rw\" (UniqueName: \"kubernetes.io/projected/1106342b-2c61-479e-bea5-1004dea87640-kube-api-access-rg6rw\") pod \"limitador-operator-controller-manager-74495c69f8-bk7js\" (UID: \"1106342b-2c61-479e-bea5-1004dea87640\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js"
Sep 18 06:33:51.615112 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:51.615082    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg6rw\" (UniqueName: \"kubernetes.io/projected/1106342b-2c61-479e-bea5-1004dea87640-kube-api-access-rg6rw\") pod \"limitador-operator-controller-manager-74495c69f8-bk7js\" (UID: \"1106342b-2c61-479e-bea5-1004dea87640\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js"
Sep 18 06:33:51.695938 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:51.695899    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js"
Sep 18 06:33:51.732678 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:51.732649    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:33:51.897773 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:51.897741    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:33:52.406266 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:52.406243    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js"]
Sep 18 06:33:52.408481 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:33:52.408457    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1106342b_2c61_479e_bea5_1004dea87640.slice/crio-77aa64ed5d20fef3a84f218a0fe420cddcf3061cbdad49421be719a072a8934b WatchSource:0}: Error finding container 77aa64ed5d20fef3a84f218a0fe420cddcf3061cbdad49421be719a072a8934b: Status 404 returned error can't find the container with id 77aa64ed5d20fef3a84f218a0fe420cddcf3061cbdad49421be719a072a8934b
Sep 18 06:33:52.480922 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:52.480887    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:33:52.722981 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:52.722906    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:33:52.822827 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:52.822798    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:33:52.927656 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:52.927626    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:33:53.026996 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:53.026925    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:33:53.222312 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:53.222276    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-q78cp" event={"ID":"3c03039b-93f8-489e-8675-9af35729e24b","Type":"ContainerStarted","Data":"f2f4ac9f6680f1c78f8a05b73c4e4229ad64ad298d1e2369c224c0ce6d9a0998"}
Sep 18 06:33:53.222498 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:53.222382    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-q78cp"
Sep 18 06:33:53.223694 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:53.223669    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js" event={"ID":"1106342b-2c61-479e-bea5-1004dea87640","Type":"ContainerStarted","Data":"77aa64ed5d20fef3a84f218a0fe420cddcf3061cbdad49421be719a072a8934b"}
Sep 18 06:33:53.255095 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:53.255038    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-q78cp" podStartSLOduration=1.777390268 podStartE2EDuration="4.255020772s" totalImagesPullingTime="2.477630504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:33:53.253342473 +0000 UTC m=+698.934870220" watchObservedRunningTime="2026-09-18 06:33:53.255020772 +0000 UTC m=+698.936548517"
Sep 18 06:33:54.229454 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:54.229414    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js" event={"ID":"1106342b-2c61-479e-bea5-1004dea87640","Type":"ContainerStarted","Data":"c0b636960397c5c987d316a4b148b3126992d28e5a9d55d0a5c11f2efad48558"}
Sep 18 06:33:54.229921 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:54.229672    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js"
Sep 18 06:33:54.246123 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:54.246074    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js" podStartSLOduration=1.84615136 podStartE2EDuration="3.246060005s" totalImagesPullingTime="1.399908645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:33:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:33:54.245046706 +0000 UTC m=+699.926574453" watchObservedRunningTime="2026-09-18 06:33:54.246060005 +0000 UTC m=+699.927587748"
Sep 18 06:33:55.709959 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:55.709927    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"]
Sep 18 06:33:55.713430 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:55.713415    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"
Sep 18 06:33:55.715971 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:55.715948    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fhtgh\""
Sep 18 06:33:55.726775 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:55.726749    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"]
Sep 18 06:33:55.834380 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:55.834348    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sbz5\" (UniqueName: \"kubernetes.io/projected/85ae0e54-1ab9-4694-95a7-aa3e794c53b8-kube-api-access-7sbz5\") pod \"kuadrant-operator-controller-manager-76f8f686bf-958td\" (UID: \"85ae0e54-1ab9-4694-95a7-aa3e794c53b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"
Sep 18 06:33:55.834514 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:55.834398    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/85ae0e54-1ab9-4694-95a7-aa3e794c53b8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-958td\" (UID: \"85ae0e54-1ab9-4694-95a7-aa3e794c53b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"
Sep 18 06:33:55.935169 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:55.935129    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sbz5\" (UniqueName: \"kubernetes.io/projected/85ae0e54-1ab9-4694-95a7-aa3e794c53b8-kube-api-access-7sbz5\") pod \"kuadrant-operator-controller-manager-76f8f686bf-958td\" (UID: \"85ae0e54-1ab9-4694-95a7-aa3e794c53b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"
Sep 18 06:33:55.935356 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:55.935196    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/85ae0e54-1ab9-4694-95a7-aa3e794c53b8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-958td\" (UID: \"85ae0e54-1ab9-4694-95a7-aa3e794c53b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"
Sep 18 06:33:55.935543 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:55.935524    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/85ae0e54-1ab9-4694-95a7-aa3e794c53b8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-958td\" (UID: \"85ae0e54-1ab9-4694-95a7-aa3e794c53b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"
Sep 18 06:33:55.943488 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:55.943459    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sbz5\" (UniqueName: \"kubernetes.io/projected/85ae0e54-1ab9-4694-95a7-aa3e794c53b8-kube-api-access-7sbz5\") pod \"kuadrant-operator-controller-manager-76f8f686bf-958td\" (UID: \"85ae0e54-1ab9-4694-95a7-aa3e794c53b8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"
Sep 18 06:33:56.022626 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:56.022536    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"
Sep 18 06:33:56.115500 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:56.115467    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:33:56.302194 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:56.302170    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"]
Sep 18 06:33:56.304194 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:33:56.304171    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85ae0e54_1ab9_4694_95a7_aa3e794c53b8.slice/crio-a6bfb1f845e3bf4d3927c01fcb6f0465f7f7332177cb3fb7ec118749c236f6f1 WatchSource:0}: Error finding container a6bfb1f845e3bf4d3927c01fcb6f0465f7f7332177cb3fb7ec118749c236f6f1: Status 404 returned error can't find the container with id a6bfb1f845e3bf4d3927c01fcb6f0465f7f7332177cb3fb7ec118749c236f6f1
Sep 18 06:33:57.120508 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:57.120435    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:33:57.243011 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:57.242974    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td" event={"ID":"85ae0e54-1ab9-4694-95a7-aa3e794c53b8","Type":"ContainerStarted","Data":"a6bfb1f845e3bf4d3927c01fcb6f0465f7f7332177cb3fb7ec118749c236f6f1"}
Sep 18 06:33:59.087096 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:59.087019    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:33:59.136173 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:59.136137    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:33:59.491913 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:59.491868    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:33:59.546383 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:33:59.546350    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:00.259967 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:00.259935    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td" event={"ID":"85ae0e54-1ab9-4694-95a7-aa3e794c53b8","Type":"ContainerStarted","Data":"9155a77a78144afb395196115df2abd518467a68955fdcd59a57fd7d47c6d8eb"}
Sep 18 06:34:00.260322 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:00.260013    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"
Sep 18 06:34:00.281681 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:00.281641    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td" podStartSLOduration=1.406039687 podStartE2EDuration="5.281628209s" totalImagesPullingTime="3.875588522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:33:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:34:00.279463174 +0000 UTC m=+705.960990917" watchObservedRunningTime="2026-09-18 06:34:00.281628209 +0000 UTC m=+705.963156006"
Sep 18 06:34:02.661407 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:02.661376    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:02.778132 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:02.778094    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:03.130030 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:03.129953    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:03.231517 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:03.231482    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:04.231817 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:04.231788    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-q78cp"
Sep 18 06:34:05.235751 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:05.235718    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js"
Sep 18 06:34:07.450906 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:07.450867    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:07.546935 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:07.546905    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:07.608955 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:07.608920    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:11.266792 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:11.266756    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"
Sep 18 06:34:12.419872 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:12.419839    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"]
Sep 18 06:34:12.426391 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:12.426371    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"
Sep 18 06:34:12.434584 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:12.434559    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"]
Sep 18 06:34:12.580979 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:12.580947    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc65x\" (UniqueName: \"kubernetes.io/projected/db54ec20-aa44-4edd-b567-37db99f181a9-kube-api-access-nc65x\") pod \"kuadrant-operator-controller-manager-5f685c496f-cln6l\" (UID: \"db54ec20-aa44-4edd-b567-37db99f181a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"
Sep 18 06:34:12.581139 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:12.580982    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db54ec20-aa44-4edd-b567-37db99f181a9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-cln6l\" (UID: \"db54ec20-aa44-4edd-b567-37db99f181a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"
Sep 18 06:34:12.682550 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:12.682473    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc65x\" (UniqueName: \"kubernetes.io/projected/db54ec20-aa44-4edd-b567-37db99f181a9-kube-api-access-nc65x\") pod \"kuadrant-operator-controller-manager-5f685c496f-cln6l\" (UID: \"db54ec20-aa44-4edd-b567-37db99f181a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"
Sep 18 06:34:12.682550 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:12.682509    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db54ec20-aa44-4edd-b567-37db99f181a9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-cln6l\" (UID: \"db54ec20-aa44-4edd-b567-37db99f181a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"
Sep 18 06:34:12.682895 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:12.682877    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db54ec20-aa44-4edd-b567-37db99f181a9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-cln6l\" (UID: \"db54ec20-aa44-4edd-b567-37db99f181a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"
Sep 18 06:34:12.690333 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:12.690316    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc65x\" (UniqueName: \"kubernetes.io/projected/db54ec20-aa44-4edd-b567-37db99f181a9-kube-api-access-nc65x\") pod \"kuadrant-operator-controller-manager-5f685c496f-cln6l\" (UID: \"db54ec20-aa44-4edd-b567-37db99f181a9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"
Sep 18 06:34:12.736691 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:12.736661    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"
Sep 18 06:34:13.016814 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.016787    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"]
Sep 18 06:34:13.017757 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:34:13.017730    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb54ec20_aa44_4edd_b567_37db99f181a9.slice/crio-cf8c70f602af148c6d14822e9b24f0192434831dac1e84b6c46b91d4708be14b WatchSource:0}: Error finding container cf8c70f602af148c6d14822e9b24f0192434831dac1e84b6c46b91d4708be14b: Status 404 returned error can't find the container with id cf8c70f602af148c6d14822e9b24f0192434831dac1e84b6c46b91d4708be14b
Sep 18 06:34:13.269678 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.269593    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"]
Sep 18 06:34:13.274302 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.274278    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"]
Sep 18 06:34:13.282399 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.282376    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"]
Sep 18 06:34:13.282584 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.282543    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td" podUID="85ae0e54-1ab9-4694-95a7-aa3e794c53b8" containerName="manager" containerID="cri-o://9155a77a78144afb395196115df2abd518467a68955fdcd59a57fd7d47c6d8eb" gracePeriod=2
Sep 18 06:34:13.296174 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.296135    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk"]
Sep 18 06:34:13.296701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.296672    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db54ec20-aa44-4edd-b567-37db99f181a9" containerName="manager"
Sep 18 06:34:13.296701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.296698    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="db54ec20-aa44-4edd-b567-37db99f181a9" containerName="manager"
Sep 18 06:34:13.296808 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.296802    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="db54ec20-aa44-4edd-b567-37db99f181a9" containerName="manager"
Sep 18 06:34:13.300335 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.300316    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"]
Sep 18 06:34:13.300439 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.300422    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk"
Sep 18 06:34:13.306894 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:13.306862    2568 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"85ae0e54-1ab9-4694-95a7-aa3e794c53b8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-958td\": pods \"kuadrant-operator-controller-manager-76f8f686bf-958td\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"
Sep 18 06:34:13.309088 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:13.309059    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-958td\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="85ae0e54-1ab9-4694-95a7-aa3e794c53b8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"
Sep 18 06:34:13.309540 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.309521    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts"]
Sep 18 06:34:13.309808 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.309798    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85ae0e54-1ab9-4694-95a7-aa3e794c53b8" containerName="manager"
Sep 18 06:34:13.309845 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.309811    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="85ae0e54-1ab9-4694-95a7-aa3e794c53b8" containerName="manager"
Sep 18 06:34:13.309881 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.309871    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="85ae0e54-1ab9-4694-95a7-aa3e794c53b8" containerName="manager"
Sep 18 06:34:13.313786 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.313768    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js"]
Sep 18 06:34:13.313985 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.313957    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js" podUID="1106342b-2c61-479e-bea5-1004dea87640" containerName="manager" containerID="cri-o://c0b636960397c5c987d316a4b148b3126992d28e5a9d55d0a5c11f2efad48558" gracePeriod=2
Sep 18 06:34:13.314295 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.314088    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts"
Sep 18 06:34:13.314385 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.314315    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk"]
Sep 18 06:34:13.320583 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.320561    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js"]
Sep 18 06:34:13.322184 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:13.322153    2568 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l" container="manager"
Sep 18 06:34:13.327565 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.327545    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts"]
Sep 18 06:34:13.340416 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:13.340382    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-cln6l\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="db54ec20-aa44-4edd-b567-37db99f181a9" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"
Sep 18 06:34:13.342154 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.342129    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-t2r5j"]
Sep 18 06:34:13.342353 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:13.342323    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-bk7js\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="1106342b-2c61-479e-bea5-1004dea87640" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js"
Sep 18 06:34:13.342502 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.342488    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1106342b-2c61-479e-bea5-1004dea87640" containerName="manager"
Sep 18 06:34:13.342502 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.342502    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1106342b-2c61-479e-bea5-1004dea87640" containerName="manager"
Sep 18 06:34:13.342620 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.342557    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1106342b-2c61-479e-bea5-1004dea87640" containerName="manager"
Sep 18 06:34:13.344345 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:13.344313    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-958td\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="85ae0e54-1ab9-4694-95a7-aa3e794c53b8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"
Sep 18 06:34:13.345763 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.345744    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-t2r5j"
Sep 18 06:34:13.355696 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.355673    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-t2r5j"]
Sep 18 06:34:13.383126 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:13.383085    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-bk7js\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="1106342b-2c61-479e-bea5-1004dea87640" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js"
Sep 18 06:34:13.402494 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:13.402465    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-958td\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="85ae0e54-1ab9-4694-95a7-aa3e794c53b8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"
Sep 18 06:34:13.404258 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:13.404234    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-cln6l\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="db54ec20-aa44-4edd-b567-37db99f181a9" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"
Sep 18 06:34:13.489144 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.488837    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f40dee7-2e9e-4f52-b13e-998f719a6fa8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hnnts\" (UID: \"3f40dee7-2e9e-4f52-b13e-998f719a6fa8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts"
Sep 18 06:34:13.489144 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.488898    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxb95\" (UniqueName: \"kubernetes.io/projected/c9b510d5-fc33-4bb6-a8ea-a6e7574305b3-kube-api-access-dxb95\") pod \"limitador-operator-controller-manager-74495c69f8-t2r5j\" (UID: \"c9b510d5-fc33-4bb6-a8ea-a6e7574305b3\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-t2r5j"
Sep 18 06:34:13.489144 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.489011    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f65g\" (UniqueName: \"kubernetes.io/projected/3f40dee7-2e9e-4f52-b13e-998f719a6fa8-kube-api-access-4f65g\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hnnts\" (UID: \"3f40dee7-2e9e-4f52-b13e-998f719a6fa8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts"
Sep 18 06:34:13.489144 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.489045    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24a14a6a-891d-49ed-8fe4-474d88127f54-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-qpgmk\" (UID: \"24a14a6a-891d-49ed-8fe4-474d88127f54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk"
Sep 18 06:34:13.489144 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.489084    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6vmq\" (UniqueName: \"kubernetes.io/projected/24a14a6a-891d-49ed-8fe4-474d88127f54-kube-api-access-f6vmq\") pod \"kuadrant-operator-controller-manager-5f685c496f-qpgmk\" (UID: \"24a14a6a-891d-49ed-8fe4-474d88127f54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk"
Sep 18 06:34:13.545200 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.545173    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js"
Sep 18 06:34:13.547521 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:13.547494    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-cln6l\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="db54ec20-aa44-4edd-b567-37db99f181a9" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"
Sep 18 06:34:13.549259 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:13.549234    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-bk7js\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="1106342b-2c61-479e-bea5-1004dea87640" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js"
Sep 18 06:34:13.550958 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:13.550914    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-958td\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="85ae0e54-1ab9-4694-95a7-aa3e794c53b8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"
Sep 18 06:34:13.553574 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.553557    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"
Sep 18 06:34:13.555336 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:13.555310    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-bk7js\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="1106342b-2c61-479e-bea5-1004dea87640" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js"
Sep 18 06:34:13.556904 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:13.556885    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-958td\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="85ae0e54-1ab9-4694-95a7-aa3e794c53b8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"
Sep 18 06:34:13.558537 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:13.558519    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-cln6l\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="db54ec20-aa44-4edd-b567-37db99f181a9" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"
Sep 18 06:34:13.590266 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.590240    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f65g\" (UniqueName: \"kubernetes.io/projected/3f40dee7-2e9e-4f52-b13e-998f719a6fa8-kube-api-access-4f65g\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hnnts\" (UID: \"3f40dee7-2e9e-4f52-b13e-998f719a6fa8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts"
Sep 18 06:34:13.590371 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.590290    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24a14a6a-891d-49ed-8fe4-474d88127f54-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-qpgmk\" (UID: \"24a14a6a-891d-49ed-8fe4-474d88127f54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk"
Sep 18 06:34:13.590371 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.590329    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6vmq\" (UniqueName: \"kubernetes.io/projected/24a14a6a-891d-49ed-8fe4-474d88127f54-kube-api-access-f6vmq\") pod \"kuadrant-operator-controller-manager-5f685c496f-qpgmk\" (UID: \"24a14a6a-891d-49ed-8fe4-474d88127f54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk"
Sep 18 06:34:13.590482 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.590375    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f40dee7-2e9e-4f52-b13e-998f719a6fa8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hnnts\" (UID: \"3f40dee7-2e9e-4f52-b13e-998f719a6fa8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts"
Sep 18 06:34:13.590482 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.590411    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxb95\" (UniqueName: \"kubernetes.io/projected/c9b510d5-fc33-4bb6-a8ea-a6e7574305b3-kube-api-access-dxb95\") pod \"limitador-operator-controller-manager-74495c69f8-t2r5j\" (UID: \"c9b510d5-fc33-4bb6-a8ea-a6e7574305b3\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-t2r5j"
Sep 18 06:34:13.590731 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.590706    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24a14a6a-891d-49ed-8fe4-474d88127f54-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-qpgmk\" (UID: \"24a14a6a-891d-49ed-8fe4-474d88127f54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk"
Sep 18 06:34:13.590780 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.590750    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f40dee7-2e9e-4f52-b13e-998f719a6fa8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hnnts\" (UID: \"3f40dee7-2e9e-4f52-b13e-998f719a6fa8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts"
Sep 18 06:34:13.597994 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.597966    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxb95\" (UniqueName: \"kubernetes.io/projected/c9b510d5-fc33-4bb6-a8ea-a6e7574305b3-kube-api-access-dxb95\") pod \"limitador-operator-controller-manager-74495c69f8-t2r5j\" (UID: \"c9b510d5-fc33-4bb6-a8ea-a6e7574305b3\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-t2r5j"
Sep 18 06:34:13.597994 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.597981    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f65g\" (UniqueName: \"kubernetes.io/projected/3f40dee7-2e9e-4f52-b13e-998f719a6fa8-kube-api-access-4f65g\") pod \"kuadrant-operator-controller-manager-76f8f686bf-hnnts\" (UID: \"3f40dee7-2e9e-4f52-b13e-998f719a6fa8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts"
Sep 18 06:34:13.598372 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.598354    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6vmq\" (UniqueName: \"kubernetes.io/projected/24a14a6a-891d-49ed-8fe4-474d88127f54-kube-api-access-f6vmq\") pod \"kuadrant-operator-controller-manager-5f685c496f-qpgmk\" (UID: \"24a14a6a-891d-49ed-8fe4-474d88127f54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk"
Sep 18 06:34:13.689814 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.689781    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk"
Sep 18 06:34:13.691028 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.691008    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/85ae0e54-1ab9-4694-95a7-aa3e794c53b8-extensions-socket-volume\") pod \"85ae0e54-1ab9-4694-95a7-aa3e794c53b8\" (UID: \"85ae0e54-1ab9-4694-95a7-aa3e794c53b8\") "
Sep 18 06:34:13.691116 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.691040    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rg6rw\" (UniqueName: \"kubernetes.io/projected/1106342b-2c61-479e-bea5-1004dea87640-kube-api-access-rg6rw\") pod \"1106342b-2c61-479e-bea5-1004dea87640\" (UID: \"1106342b-2c61-479e-bea5-1004dea87640\") "
Sep 18 06:34:13.691116 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.691066    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7sbz5\" (UniqueName: \"kubernetes.io/projected/85ae0e54-1ab9-4694-95a7-aa3e794c53b8-kube-api-access-7sbz5\") pod \"85ae0e54-1ab9-4694-95a7-aa3e794c53b8\" (UID: \"85ae0e54-1ab9-4694-95a7-aa3e794c53b8\") "
Sep 18 06:34:13.691627 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.691560    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85ae0e54-1ab9-4694-95a7-aa3e794c53b8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "85ae0e54-1ab9-4694-95a7-aa3e794c53b8" (UID: "85ae0e54-1ab9-4694-95a7-aa3e794c53b8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:34:13.693391 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.693367    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1106342b-2c61-479e-bea5-1004dea87640-kube-api-access-rg6rw" (OuterVolumeSpecName: "kube-api-access-rg6rw") pod "1106342b-2c61-479e-bea5-1004dea87640" (UID: "1106342b-2c61-479e-bea5-1004dea87640"). InnerVolumeSpecName "kube-api-access-rg6rw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:34:13.693517 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.693496    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85ae0e54-1ab9-4694-95a7-aa3e794c53b8-kube-api-access-7sbz5" (OuterVolumeSpecName: "kube-api-access-7sbz5") pod "85ae0e54-1ab9-4694-95a7-aa3e794c53b8" (UID: "85ae0e54-1ab9-4694-95a7-aa3e794c53b8"). InnerVolumeSpecName "kube-api-access-7sbz5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:34:13.698852 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.698751    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts"
Sep 18 06:34:13.705991 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.705968    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-t2r5j"
Sep 18 06:34:13.792745 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.792680    2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/85ae0e54-1ab9-4694-95a7-aa3e794c53b8-extensions-socket-volume\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:34:13.792745 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.792709    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rg6rw\" (UniqueName: \"kubernetes.io/projected/1106342b-2c61-479e-bea5-1004dea87640-kube-api-access-rg6rw\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:34:13.792745 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.792723    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7sbz5\" (UniqueName: \"kubernetes.io/projected/85ae0e54-1ab9-4694-95a7-aa3e794c53b8-kube-api-access-7sbz5\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:34:13.955532 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.955503    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:13.978150 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.978012    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk"]
Sep 18 06:34:13.980875 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:34:13.980844    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24a14a6a_891d_49ed_8fe4_474d88127f54.slice/crio-f20c4c3765c56c61e3b257ce1885c4ce055720b5f82c63ddbfc64800210dada5 WatchSource:0}: Error finding container f20c4c3765c56c61e3b257ce1885c4ce055720b5f82c63ddbfc64800210dada5: Status 404 returned error can't find the container with id f20c4c3765c56c61e3b257ce1885c4ce055720b5f82c63ddbfc64800210dada5
Sep 18 06:34:13.989695 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.989674    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-t2r5j"]
Sep 18 06:34:13.994280 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:13.994208    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts"]
Sep 18 06:34:13.999554 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:34:13.999486    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f40dee7_2e9e_4f52_b13e_998f719a6fa8.slice/crio-7fe30b97d49b34a90aa961a6b8aa1b5cb051e73cd9853ca27208a7186be1c329 WatchSource:0}: Error finding container 7fe30b97d49b34a90aa961a6b8aa1b5cb051e73cd9853ca27208a7186be1c329: Status 404 returned error can't find the container with id 7fe30b97d49b34a90aa961a6b8aa1b5cb051e73cd9853ca27208a7186be1c329
Sep 18 06:34:14.049246 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.049191    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:14.328620 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.328531    2568 generic.go:356] "Generic (PLEG): container finished" podID="85ae0e54-1ab9-4694-95a7-aa3e794c53b8" containerID="9155a77a78144afb395196115df2abd518467a68955fdcd59a57fd7d47c6d8eb" exitCode=0
Sep 18 06:34:14.328620 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.328602    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"
Sep 18 06:34:14.328835 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.328625    2568 scope.go:117] "RemoveContainer" containerID="9155a77a78144afb395196115df2abd518467a68955fdcd59a57fd7d47c6d8eb"
Sep 18 06:34:14.330410 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.330382    2568 generic.go:356] "Generic (PLEG): container finished" podID="1106342b-2c61-479e-bea5-1004dea87640" containerID="c0b636960397c5c987d316a4b148b3126992d28e5a9d55d0a5c11f2efad48558" exitCode=0
Sep 18 06:34:14.330557 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.330510    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js"
Sep 18 06:34:14.331013 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:14.330982    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-bk7js\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="1106342b-2c61-479e-bea5-1004dea87640" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js"
Sep 18 06:34:14.332917 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:14.332878    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-958td\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="85ae0e54-1ab9-4694-95a7-aa3e794c53b8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"
Sep 18 06:34:14.335886 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:14.335855    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-cln6l\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="db54ec20-aa44-4edd-b567-37db99f181a9" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"
Sep 18 06:34:14.336677 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.336630    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts" event={"ID":"3f40dee7-2e9e-4f52-b13e-998f719a6fa8","Type":"ContainerStarted","Data":"b11e099d9bf2440691b3380f3c1f6a3178ae84efc443a514a03b755e4b061f6f"}
Sep 18 06:34:14.336812 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.336716    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts"
Sep 18 06:34:14.336812 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.336742    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts" event={"ID":"3f40dee7-2e9e-4f52-b13e-998f719a6fa8","Type":"ContainerStarted","Data":"7fe30b97d49b34a90aa961a6b8aa1b5cb051e73cd9853ca27208a7186be1c329"}
Sep 18 06:34:14.337851 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:14.337792    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-cln6l\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="db54ec20-aa44-4edd-b567-37db99f181a9" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"
Sep 18 06:34:14.339836 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.338895    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l" podUID="db54ec20-aa44-4edd-b567-37db99f181a9" containerName="manager" containerID="cri-o://62150d0d3e266c373f532214c3177f091fba6fcbd99dc6632fb4852154d91178" gracePeriod=2
Sep 18 06:34:14.339836 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:14.339640    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-bk7js\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="1106342b-2c61-479e-bea5-1004dea87640" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js"
Sep 18 06:34:14.342720 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:14.342390    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-958td\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="85ae0e54-1ab9-4694-95a7-aa3e794c53b8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"
Sep 18 06:34:14.343453 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.342972    2568 scope.go:117] "RemoveContainer" containerID="9155a77a78144afb395196115df2abd518467a68955fdcd59a57fd7d47c6d8eb"
Sep 18 06:34:14.343587 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:14.343555    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9155a77a78144afb395196115df2abd518467a68955fdcd59a57fd7d47c6d8eb\": container with ID starting with 9155a77a78144afb395196115df2abd518467a68955fdcd59a57fd7d47c6d8eb not found: ID does not exist" containerID="9155a77a78144afb395196115df2abd518467a68955fdcd59a57fd7d47c6d8eb"
Sep 18 06:34:14.343714 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.343597    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9155a77a78144afb395196115df2abd518467a68955fdcd59a57fd7d47c6d8eb"} err="failed to get container status \"9155a77a78144afb395196115df2abd518467a68955fdcd59a57fd7d47c6d8eb\": rpc error: code = NotFound desc = could not find container \"9155a77a78144afb395196115df2abd518467a68955fdcd59a57fd7d47c6d8eb\": container with ID starting with 9155a77a78144afb395196115df2abd518467a68955fdcd59a57fd7d47c6d8eb not found: ID does not exist"
Sep 18 06:34:14.343714 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.343622    2568 scope.go:117] "RemoveContainer" containerID="c0b636960397c5c987d316a4b148b3126992d28e5a9d55d0a5c11f2efad48558"
Sep 18 06:34:14.343847 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.343710    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-t2r5j" event={"ID":"c9b510d5-fc33-4bb6-a8ea-a6e7574305b3","Type":"ContainerStarted","Data":"b2ae930d72b79dfc424ab05ed8a95de10ccb5130ff75345308a50afdee30c11a"}
Sep 18 06:34:14.343847 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.343767    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-t2r5j" event={"ID":"c9b510d5-fc33-4bb6-a8ea-a6e7574305b3","Type":"ContainerStarted","Data":"0aec133dfc99f81da715553f1d807f616c152fea2e50846ca0b078cec2b12c1f"}
Sep 18 06:34:14.343847 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.343798    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-t2r5j"
Sep 18 06:34:14.345987 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.345959    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk" event={"ID":"24a14a6a-891d-49ed-8fe4-474d88127f54","Type":"ContainerStarted","Data":"8a5130bff161249f4a9c488cbad4c227e1730be4726c365b346b9575fbf458f9"}
Sep 18 06:34:14.346089 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.346021    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk"
Sep 18 06:34:14.346089 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.346047    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk" event={"ID":"24a14a6a-891d-49ed-8fe4-474d88127f54","Type":"ContainerStarted","Data":"f20c4c3765c56c61e3b257ce1885c4ce055720b5f82c63ddbfc64800210dada5"}
Sep 18 06:34:14.352478 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.352461    2568 scope.go:117] "RemoveContainer" containerID="c0b636960397c5c987d316a4b148b3126992d28e5a9d55d0a5c11f2efad48558"
Sep 18 06:34:14.352722 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:14.352704    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0b636960397c5c987d316a4b148b3126992d28e5a9d55d0a5c11f2efad48558\": container with ID starting with c0b636960397c5c987d316a4b148b3126992d28e5a9d55d0a5c11f2efad48558 not found: ID does not exist" containerID="c0b636960397c5c987d316a4b148b3126992d28e5a9d55d0a5c11f2efad48558"
Sep 18 06:34:14.352775 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.352729    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0b636960397c5c987d316a4b148b3126992d28e5a9d55d0a5c11f2efad48558"} err="failed to get container status \"c0b636960397c5c987d316a4b148b3126992d28e5a9d55d0a5c11f2efad48558\": rpc error: code = NotFound desc = could not find container \"c0b636960397c5c987d316a4b148b3126992d28e5a9d55d0a5c11f2efad48558\": container with ID starting with c0b636960397c5c987d316a4b148b3126992d28e5a9d55d0a5c11f2efad48558 not found: ID does not exist"
Sep 18 06:34:14.361787 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.361732    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts" podStartSLOduration=1.361719023 podStartE2EDuration="1.361719023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:34:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:34:14.359978992 +0000 UTC m=+720.041506728" watchObservedRunningTime="2026-09-18 06:34:14.361719023 +0000 UTC m=+720.043246858"
Sep 18 06:34:14.361890 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:14.361836    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-cln6l\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="db54ec20-aa44-4edd-b567-37db99f181a9" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"
Sep 18 06:34:14.363629 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:14.363608    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-bk7js\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="1106342b-2c61-479e-bea5-1004dea87640" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js"
Sep 18 06:34:14.365378 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:14.365356    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-958td\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="85ae0e54-1ab9-4694-95a7-aa3e794c53b8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"
Sep 18 06:34:14.367387 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:14.367366    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-958td\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="85ae0e54-1ab9-4694-95a7-aa3e794c53b8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-958td"
Sep 18 06:34:14.387049 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.387013    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk" podStartSLOduration=1.3870012250000001 podStartE2EDuration="1.387001225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:34:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:34:14.385709525 +0000 UTC m=+720.067237266" watchObservedRunningTime="2026-09-18 06:34:14.387001225 +0000 UTC m=+720.068528968"
Sep 18 06:34:14.387588 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:14.387566    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-cln6l\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="db54ec20-aa44-4edd-b567-37db99f181a9" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"
Sep 18 06:34:14.389296 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:14.389277    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-bk7js\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="1106342b-2c61-479e-bea5-1004dea87640" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-bk7js"
Sep 18 06:34:14.410548 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.410509    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-t2r5j" podStartSLOduration=1.41049669 podStartE2EDuration="1.41049669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:34:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:34:14.408135928 +0000 UTC m=+720.089663671" watchObservedRunningTime="2026-09-18 06:34:14.41049669 +0000 UTC m=+720.092024433"
Sep 18 06:34:14.543454 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.543419    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1106342b-2c61-479e-bea5-1004dea87640" path="/var/lib/kubelet/pods/1106342b-2c61-479e-bea5-1004dea87640/volumes"
Sep 18 06:34:14.543944 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.543924    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85ae0e54-1ab9-4694-95a7-aa3e794c53b8" path="/var/lib/kubelet/pods/85ae0e54-1ab9-4694-95a7-aa3e794c53b8/volumes"
Sep 18 06:34:14.567761 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.567736    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"
Sep 18 06:34:14.597616 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.597562    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nc65x\" (UniqueName: \"kubernetes.io/projected/db54ec20-aa44-4edd-b567-37db99f181a9-kube-api-access-nc65x\") pod \"db54ec20-aa44-4edd-b567-37db99f181a9\" (UID: \"db54ec20-aa44-4edd-b567-37db99f181a9\") "
Sep 18 06:34:14.597616 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.597596    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db54ec20-aa44-4edd-b567-37db99f181a9-extensions-socket-volume\") pod \"db54ec20-aa44-4edd-b567-37db99f181a9\" (UID: \"db54ec20-aa44-4edd-b567-37db99f181a9\") "
Sep 18 06:34:14.597975 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.597952    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db54ec20-aa44-4edd-b567-37db99f181a9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "db54ec20-aa44-4edd-b567-37db99f181a9" (UID: "db54ec20-aa44-4edd-b567-37db99f181a9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:34:14.599586 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.599566    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db54ec20-aa44-4edd-b567-37db99f181a9-kube-api-access-nc65x" (OuterVolumeSpecName: "kube-api-access-nc65x") pod "db54ec20-aa44-4edd-b567-37db99f181a9" (UID: "db54ec20-aa44-4edd-b567-37db99f181a9"). InnerVolumeSpecName "kube-api-access-nc65x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:34:14.698266 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.698212    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nc65x\" (UniqueName: \"kubernetes.io/projected/db54ec20-aa44-4edd-b567-37db99f181a9-kube-api-access-nc65x\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:34:14.698382 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:14.698270    2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db54ec20-aa44-4edd-b567-37db99f181a9-extensions-socket-volume\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:34:15.354400 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:15.354364    2568 generic.go:356] "Generic (PLEG): container finished" podID="db54ec20-aa44-4edd-b567-37db99f181a9" containerID="62150d0d3e266c373f532214c3177f091fba6fcbd99dc6632fb4852154d91178" exitCode=0
Sep 18 06:34:15.354607 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:15.354407    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"
Sep 18 06:34:15.354607 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:15.354455    2568 scope.go:117] "RemoveContainer" containerID="62150d0d3e266c373f532214c3177f091fba6fcbd99dc6632fb4852154d91178"
Sep 18 06:34:15.357132 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:15.357106    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-cln6l\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="db54ec20-aa44-4edd-b567-37db99f181a9" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"
Sep 18 06:34:15.363054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:15.363039    2568 scope.go:117] "RemoveContainer" containerID="62150d0d3e266c373f532214c3177f091fba6fcbd99dc6632fb4852154d91178"
Sep 18 06:34:15.363333 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:15.363314    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62150d0d3e266c373f532214c3177f091fba6fcbd99dc6632fb4852154d91178\": container with ID starting with 62150d0d3e266c373f532214c3177f091fba6fcbd99dc6632fb4852154d91178 not found: ID does not exist" containerID="62150d0d3e266c373f532214c3177f091fba6fcbd99dc6632fb4852154d91178"
Sep 18 06:34:15.363379 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:15.363340    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62150d0d3e266c373f532214c3177f091fba6fcbd99dc6632fb4852154d91178"} err="failed to get container status \"62150d0d3e266c373f532214c3177f091fba6fcbd99dc6632fb4852154d91178\": rpc error: code = NotFound desc = could not find container \"62150d0d3e266c373f532214c3177f091fba6fcbd99dc6632fb4852154d91178\": container with ID starting with 62150d0d3e266c373f532214c3177f091fba6fcbd99dc6632fb4852154d91178 not found: ID does not exist"
Sep 18 06:34:15.364788 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:15.364765    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-cln6l\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="db54ec20-aa44-4edd-b567-37db99f181a9" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"
Sep 18 06:34:16.541282 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:16.541239    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-cln6l\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="db54ec20-aa44-4edd-b567-37db99f181a9" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-cln6l"
Sep 18 06:34:16.543080 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:16.543058    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db54ec20-aa44-4edd-b567-37db99f181a9" path="/var/lib/kubelet/pods/db54ec20-aa44-4edd-b567-37db99f181a9/volumes"
Sep 18 06:34:19.665112 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:19.665078    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:19.766106 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:19.766073    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:20.268703 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:20.268668    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:20.323579 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:20.323547    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:23.670568 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:23.670533    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:23.765790 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:23.765755    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:23.866073 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:23.866046    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:25.356883 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.356853    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk"
Sep 18 06:34:25.357270 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.356927    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-t2r5j"
Sep 18 06:34:25.357270 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.357080    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts"
Sep 18 06:34:25.408784 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.408757    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts"]
Sep 18 06:34:25.408938 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.408921    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts" podUID="3f40dee7-2e9e-4f52-b13e-998f719a6fa8" containerName="manager" containerID="cri-o://b11e099d9bf2440691b3380f3c1f6a3178ae84efc443a514a03b755e4b061f6f" gracePeriod=10
Sep 18 06:34:25.655791 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.655767    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts"
Sep 18 06:34:25.691992 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.691964    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4f65g\" (UniqueName: \"kubernetes.io/projected/3f40dee7-2e9e-4f52-b13e-998f719a6fa8-kube-api-access-4f65g\") pod \"3f40dee7-2e9e-4f52-b13e-998f719a6fa8\" (UID: \"3f40dee7-2e9e-4f52-b13e-998f719a6fa8\") "
Sep 18 06:34:25.692166 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.692063    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f40dee7-2e9e-4f52-b13e-998f719a6fa8-extensions-socket-volume\") pod \"3f40dee7-2e9e-4f52-b13e-998f719a6fa8\" (UID: \"3f40dee7-2e9e-4f52-b13e-998f719a6fa8\") "
Sep 18 06:34:25.692523 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.692494    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f40dee7-2e9e-4f52-b13e-998f719a6fa8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3f40dee7-2e9e-4f52-b13e-998f719a6fa8" (UID: "3f40dee7-2e9e-4f52-b13e-998f719a6fa8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:34:25.694476 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.694443    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f40dee7-2e9e-4f52-b13e-998f719a6fa8-kube-api-access-4f65g" (OuterVolumeSpecName: "kube-api-access-4f65g") pod "3f40dee7-2e9e-4f52-b13e-998f719a6fa8" (UID: "3f40dee7-2e9e-4f52-b13e-998f719a6fa8"). InnerVolumeSpecName "kube-api-access-4f65g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:34:25.737707 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.737670    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq"]
Sep 18 06:34:25.738086 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.738069    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f40dee7-2e9e-4f52-b13e-998f719a6fa8" containerName="manager"
Sep 18 06:34:25.738167 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.738089    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f40dee7-2e9e-4f52-b13e-998f719a6fa8" containerName="manager"
Sep 18 06:34:25.738236 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.738176    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f40dee7-2e9e-4f52-b13e-998f719a6fa8" containerName="manager"
Sep 18 06:34:25.740405 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.740386    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq"
Sep 18 06:34:25.752375 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.752351    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq"]
Sep 18 06:34:25.793024 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.792997    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/de65d394-4e57-415d-8b9c-42dfb499258c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zqxbq\" (UID: \"de65d394-4e57-415d-8b9c-42dfb499258c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq"
Sep 18 06:34:25.793164 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.793144    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sngm\" (UniqueName: \"kubernetes.io/projected/de65d394-4e57-415d-8b9c-42dfb499258c-kube-api-access-5sngm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zqxbq\" (UID: \"de65d394-4e57-415d-8b9c-42dfb499258c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq"
Sep 18 06:34:25.793248 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.793186    2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3f40dee7-2e9e-4f52-b13e-998f719a6fa8-extensions-socket-volume\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:34:25.793248 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.793196    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4f65g\" (UniqueName: \"kubernetes.io/projected/3f40dee7-2e9e-4f52-b13e-998f719a6fa8-kube-api-access-4f65g\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:34:25.894039 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.893961    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sngm\" (UniqueName: \"kubernetes.io/projected/de65d394-4e57-415d-8b9c-42dfb499258c-kube-api-access-5sngm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zqxbq\" (UID: \"de65d394-4e57-415d-8b9c-42dfb499258c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq"
Sep 18 06:34:25.894039 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.893994    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/de65d394-4e57-415d-8b9c-42dfb499258c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zqxbq\" (UID: \"de65d394-4e57-415d-8b9c-42dfb499258c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq"
Sep 18 06:34:25.894377 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.894358    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/de65d394-4e57-415d-8b9c-42dfb499258c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zqxbq\" (UID: \"de65d394-4e57-415d-8b9c-42dfb499258c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq"
Sep 18 06:34:25.907844 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:25.907820    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sngm\" (UniqueName: \"kubernetes.io/projected/de65d394-4e57-415d-8b9c-42dfb499258c-kube-api-access-5sngm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-zqxbq\" (UID: \"de65d394-4e57-415d-8b9c-42dfb499258c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq"
Sep 18 06:34:26.050056 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:26.050018    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq"
Sep 18 06:34:26.323601 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:26.323574    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq"]
Sep 18 06:34:26.325268 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:34:26.325242    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde65d394_4e57_415d_8b9c_42dfb499258c.slice/crio-e3e009fe21177100b64c227b675c8d7cc2f753531bb8e6aee448c0353b5de002 WatchSource:0}: Error finding container e3e009fe21177100b64c227b675c8d7cc2f753531bb8e6aee448c0353b5de002: Status 404 returned error can't find the container with id e3e009fe21177100b64c227b675c8d7cc2f753531bb8e6aee448c0353b5de002
Sep 18 06:34:26.410548 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:26.410522    2568 generic.go:356] "Generic (PLEG): container finished" podID="3f40dee7-2e9e-4f52-b13e-998f719a6fa8" containerID="b11e099d9bf2440691b3380f3c1f6a3178ae84efc443a514a03b755e4b061f6f" exitCode=0
Sep 18 06:34:26.410879 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:26.410564    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts"
Sep 18 06:34:26.410879 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:26.410610    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts" event={"ID":"3f40dee7-2e9e-4f52-b13e-998f719a6fa8","Type":"ContainerDied","Data":"b11e099d9bf2440691b3380f3c1f6a3178ae84efc443a514a03b755e4b061f6f"}
Sep 18 06:34:26.410879 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:26.410659    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts" event={"ID":"3f40dee7-2e9e-4f52-b13e-998f719a6fa8","Type":"ContainerDied","Data":"7fe30b97d49b34a90aa961a6b8aa1b5cb051e73cd9853ca27208a7186be1c329"}
Sep 18 06:34:26.410879 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:26.410686    2568 scope.go:117] "RemoveContainer" containerID="b11e099d9bf2440691b3380f3c1f6a3178ae84efc443a514a03b755e4b061f6f"
Sep 18 06:34:26.412514 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:26.412493    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq" event={"ID":"de65d394-4e57-415d-8b9c-42dfb499258c","Type":"ContainerStarted","Data":"695e2697c31563983d95fa57bd26489b75b52e89cb1d0202ddcdd176a31acdcb"}
Sep 18 06:34:26.412646 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:26.412540    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq" event={"ID":"de65d394-4e57-415d-8b9c-42dfb499258c","Type":"ContainerStarted","Data":"e3e009fe21177100b64c227b675c8d7cc2f753531bb8e6aee448c0353b5de002"}
Sep 18 06:34:26.412646 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:26.412579    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq"
Sep 18 06:34:26.418968 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:26.418894    2568 scope.go:117] "RemoveContainer" containerID="b11e099d9bf2440691b3380f3c1f6a3178ae84efc443a514a03b755e4b061f6f"
Sep 18 06:34:26.419258 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:26.419238    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b11e099d9bf2440691b3380f3c1f6a3178ae84efc443a514a03b755e4b061f6f\": container with ID starting with b11e099d9bf2440691b3380f3c1f6a3178ae84efc443a514a03b755e4b061f6f not found: ID does not exist" containerID="b11e099d9bf2440691b3380f3c1f6a3178ae84efc443a514a03b755e4b061f6f"
Sep 18 06:34:26.419340 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:26.419269    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b11e099d9bf2440691b3380f3c1f6a3178ae84efc443a514a03b755e4b061f6f"} err="failed to get container status \"b11e099d9bf2440691b3380f3c1f6a3178ae84efc443a514a03b755e4b061f6f\": rpc error: code = NotFound desc = could not find container \"b11e099d9bf2440691b3380f3c1f6a3178ae84efc443a514a03b755e4b061f6f\": container with ID starting with b11e099d9bf2440691b3380f3c1f6a3178ae84efc443a514a03b755e4b061f6f not found: ID does not exist"
Sep 18 06:34:26.438051 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:26.438016    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq" podStartSLOduration=1.438002072 podStartE2EDuration="1.438002072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:34:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:34:26.436932913 +0000 UTC m=+732.118460659" watchObservedRunningTime="2026-09-18 06:34:26.438002072 +0000 UTC m=+732.119529820"
Sep 18 06:34:26.462200 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:26.462176    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts"]
Sep 18 06:34:26.467694 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:26.467674    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-hnnts"]
Sep 18 06:34:26.546142 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:26.546101    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f40dee7-2e9e-4f52-b13e-998f719a6fa8" path="/var/lib/kubelet/pods/3f40dee7-2e9e-4f52-b13e-998f719a6fa8/volumes"
Sep 18 06:34:27.676668 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:27.676613    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:27.780504 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:27.780469    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:27.876697 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:27.876659    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:27.978488 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:27.978401    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:28.081236 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:28.081179    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:28.186187 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:28.186155    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:33.931882 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:33.931846    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:33.986326 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:33.986293    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:37.421007 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:37.420981    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq"
Sep 18 06:34:37.482452 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:37.482412    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk"]
Sep 18 06:34:37.482795 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:37.482766    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk" podUID="24a14a6a-891d-49ed-8fe4-474d88127f54" containerName="manager" containerID="cri-o://8a5130bff161249f4a9c488cbad4c227e1730be4726c365b346b9575fbf458f9" gracePeriod=10
Sep 18 06:34:37.725036 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:37.725009    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk"
Sep 18 06:34:37.793877 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:37.793848    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6vmq\" (UniqueName: \"kubernetes.io/projected/24a14a6a-891d-49ed-8fe4-474d88127f54-kube-api-access-f6vmq\") pod \"24a14a6a-891d-49ed-8fe4-474d88127f54\" (UID: \"24a14a6a-891d-49ed-8fe4-474d88127f54\") "
Sep 18 06:34:37.794054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:37.793891    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24a14a6a-891d-49ed-8fe4-474d88127f54-extensions-socket-volume\") pod \"24a14a6a-891d-49ed-8fe4-474d88127f54\" (UID: \"24a14a6a-891d-49ed-8fe4-474d88127f54\") "
Sep 18 06:34:37.794361 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:37.794339    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24a14a6a-891d-49ed-8fe4-474d88127f54-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "24a14a6a-891d-49ed-8fe4-474d88127f54" (UID: "24a14a6a-891d-49ed-8fe4-474d88127f54"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:34:37.796017 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:37.795999    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24a14a6a-891d-49ed-8fe4-474d88127f54-kube-api-access-f6vmq" (OuterVolumeSpecName: "kube-api-access-f6vmq") pod "24a14a6a-891d-49ed-8fe4-474d88127f54" (UID: "24a14a6a-891d-49ed-8fe4-474d88127f54"). InnerVolumeSpecName "kube-api-access-f6vmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:34:37.895105 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:37.895072    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6vmq\" (UniqueName: \"kubernetes.io/projected/24a14a6a-891d-49ed-8fe4-474d88127f54-kube-api-access-f6vmq\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:34:37.895105 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:37.895099    2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/24a14a6a-891d-49ed-8fe4-474d88127f54-extensions-socket-volume\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:34:38.466160 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:38.466121    2568 generic.go:356] "Generic (PLEG): container finished" podID="24a14a6a-891d-49ed-8fe4-474d88127f54" containerID="8a5130bff161249f4a9c488cbad4c227e1730be4726c365b346b9575fbf458f9" exitCode=0
Sep 18 06:34:38.466667 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:38.466170    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk"
Sep 18 06:34:38.466667 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:38.466211    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk" event={"ID":"24a14a6a-891d-49ed-8fe4-474d88127f54","Type":"ContainerDied","Data":"8a5130bff161249f4a9c488cbad4c227e1730be4726c365b346b9575fbf458f9"}
Sep 18 06:34:38.466667 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:38.466285    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk" event={"ID":"24a14a6a-891d-49ed-8fe4-474d88127f54","Type":"ContainerDied","Data":"f20c4c3765c56c61e3b257ce1885c4ce055720b5f82c63ddbfc64800210dada5"}
Sep 18 06:34:38.466667 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:38.466307    2568 scope.go:117] "RemoveContainer" containerID="8a5130bff161249f4a9c488cbad4c227e1730be4726c365b346b9575fbf458f9"
Sep 18 06:34:38.474759 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:38.474747    2568 scope.go:117] "RemoveContainer" containerID="8a5130bff161249f4a9c488cbad4c227e1730be4726c365b346b9575fbf458f9"
Sep 18 06:34:38.475021 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:34:38.475005    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a5130bff161249f4a9c488cbad4c227e1730be4726c365b346b9575fbf458f9\": container with ID starting with 8a5130bff161249f4a9c488cbad4c227e1730be4726c365b346b9575fbf458f9 not found: ID does not exist" containerID="8a5130bff161249f4a9c488cbad4c227e1730be4726c365b346b9575fbf458f9"
Sep 18 06:34:38.475067 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:38.475027    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a5130bff161249f4a9c488cbad4c227e1730be4726c365b346b9575fbf458f9"} err="failed to get container status \"8a5130bff161249f4a9c488cbad4c227e1730be4726c365b346b9575fbf458f9\": rpc error: code = NotFound desc = could not find container \"8a5130bff161249f4a9c488cbad4c227e1730be4726c365b346b9575fbf458f9\": container with ID starting with 8a5130bff161249f4a9c488cbad4c227e1730be4726c365b346b9575fbf458f9 not found: ID does not exist"
Sep 18 06:34:38.489161 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:38.489132    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk"]
Sep 18 06:34:38.496545 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:38.496520    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-qpgmk"]
Sep 18 06:34:38.542162 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:38.542122    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24a14a6a-891d-49ed-8fe4-474d88127f54" path="/var/lib/kubelet/pods/24a14a6a-891d-49ed-8fe4-474d88127f54/volumes"
Sep 18 06:34:42.319535 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.319501    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"]
Sep 18 06:34:42.319899 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.319849    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24a14a6a-891d-49ed-8fe4-474d88127f54" containerName="manager"
Sep 18 06:34:42.319899 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.319860    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="24a14a6a-891d-49ed-8fe4-474d88127f54" containerName="manager"
Sep 18 06:34:42.319976 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.319920    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="24a14a6a-891d-49ed-8fe4-474d88127f54" containerName="manager"
Sep 18 06:34:42.324526 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.324506    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.327288 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.327263    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 18 06:34:42.327288 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.327286    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-9pqvp\""
Sep 18 06:34:42.333238 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.333191    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"]
Sep 18 06:34:42.350885 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.350864    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:42.434522 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.434493    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0484017e-ee10-41d4-b810-b3a80f6382da-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.434653 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.434525    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0484017e-ee10-41d4-b810-b3a80f6382da-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.434653 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.434547    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0484017e-ee10-41d4-b810-b3a80f6382da-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.434653 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.434569    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0484017e-ee10-41d4-b810-b3a80f6382da-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.434653 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.434631    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0484017e-ee10-41d4-b810-b3a80f6382da-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.434792 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.434688    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0484017e-ee10-41d4-b810-b3a80f6382da-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.434792 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.434716    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0484017e-ee10-41d4-b810-b3a80f6382da-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.434792 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.434749    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0484017e-ee10-41d4-b810-b3a80f6382da-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.434792 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.434765    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqvt7\" (UniqueName: \"kubernetes.io/projected/0484017e-ee10-41d4-b810-b3a80f6382da-kube-api-access-nqvt7\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.487099 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.487074    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:42.535788 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.535762    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0484017e-ee10-41d4-b810-b3a80f6382da-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.535788 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.535792    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0484017e-ee10-41d4-b810-b3a80f6382da-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.536011 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.535814    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0484017e-ee10-41d4-b810-b3a80f6382da-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.536011 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.535834    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0484017e-ee10-41d4-b810-b3a80f6382da-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.536011 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.535885    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0484017e-ee10-41d4-b810-b3a80f6382da-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.536011 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.535933    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0484017e-ee10-41d4-b810-b3a80f6382da-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.536011 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.535962    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0484017e-ee10-41d4-b810-b3a80f6382da-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.536011 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.535998    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0484017e-ee10-41d4-b810-b3a80f6382da-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.536347 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.536023    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqvt7\" (UniqueName: \"kubernetes.io/projected/0484017e-ee10-41d4-b810-b3a80f6382da-kube-api-access-nqvt7\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.536347 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.536297    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0484017e-ee10-41d4-b810-b3a80f6382da-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.536347 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.536321    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0484017e-ee10-41d4-b810-b3a80f6382da-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.536503 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.536419    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0484017e-ee10-41d4-b810-b3a80f6382da-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.536576 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.536547    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0484017e-ee10-41d4-b810-b3a80f6382da-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.536763 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.536742    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0484017e-ee10-41d4-b810-b3a80f6382da-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.538335 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.538315    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0484017e-ee10-41d4-b810-b3a80f6382da-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.538977 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.538954    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0484017e-ee10-41d4-b810-b3a80f6382da-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.543146 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.543124    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0484017e-ee10-41d4-b810-b3a80f6382da-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.543353 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.543331    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqvt7\" (UniqueName: \"kubernetes.io/projected/0484017e-ee10-41d4-b810-b3a80f6382da-kube-api-access-nqvt7\") pod \"maas-default-gateway-openshift-default-78bb78ff78-f2pj4\" (UID: \"0484017e-ee10-41d4-b810-b3a80f6382da\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.637121 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.637042    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:42.906432 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:42.906404    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"]
Sep 18 06:34:42.909235 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:34:42.909189    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0484017e_ee10_41d4_b810_b3a80f6382da.slice/crio-11df02d783caf7775a952225a574a734f619b8c658534b553e5405d9957dc4db WatchSource:0}: Error finding container 11df02d783caf7775a952225a574a734f619b8c658534b553e5405d9957dc4db: Status 404 returned error can't find the container with id 11df02d783caf7775a952225a574a734f619b8c658534b553e5405d9957dc4db
Sep 18 06:34:43.490366 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:43.490333    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4" event={"ID":"0484017e-ee10-41d4-b810-b3a80f6382da","Type":"ContainerStarted","Data":"11df02d783caf7775a952225a574a734f619b8c658534b553e5405d9957dc4db"}
Sep 18 06:34:45.446005 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:45.445966    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:34:45.446410 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:45.446066    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:34:45.446410 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:45.446124    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:34:46.513995 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:46.513962    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4" event={"ID":"0484017e-ee10-41d4-b810-b3a80f6382da","Type":"ContainerStarted","Data":"6761e3acbfb5c7c7c74bc9c6769482a8985f0a34ef7651aee9ce7ab49124cc7d"}
Sep 18 06:34:46.533093 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:46.533048    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4" podStartSLOduration=1.998630678 podStartE2EDuration="4.533032422s" totalImagesPullingTime="2.534401744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:34:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:34:46.531193666 +0000 UTC m=+752.212721410" watchObservedRunningTime="2026-09-18 06:34:46.533032422 +0000 UTC m=+752.214560166"
Sep 18 06:34:46.637190 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:46.637159    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:46.646665 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:46.646633    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:47.519018 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:47.518987    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:47.519819 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:47.519802    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:34:55.662056 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:55.662010    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:56.721298 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:56.721252    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:56.832356 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:56.832326    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:56.932248 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:56.932202    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:57.122724 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.122641    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:34:57.126414 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.126393    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-sczjl"
Sep 18 06:34:57.128645 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.128628    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-p79c2\""
Sep 18 06:34:57.128844 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.128829    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 18 06:34:57.132791 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.132771    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:34:57.145304 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.145275    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:34:57.264991 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.264955    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hgjl\" (UniqueName: \"kubernetes.io/projected/fa87303d-4f34-4bb3-9cf6-9c53882f33eb-kube-api-access-2hgjl\") pod \"limitador-limitador-84994f7ffc-sczjl\" (UID: \"fa87303d-4f34-4bb3-9cf6-9c53882f33eb\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-sczjl"
Sep 18 06:34:57.265175 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.264999    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fa87303d-4f34-4bb3-9cf6-9c53882f33eb-config-file\") pod \"limitador-limitador-84994f7ffc-sczjl\" (UID: \"fa87303d-4f34-4bb3-9cf6-9c53882f33eb\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-sczjl"
Sep 18 06:34:57.365547 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.365518    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hgjl\" (UniqueName: \"kubernetes.io/projected/fa87303d-4f34-4bb3-9cf6-9c53882f33eb-kube-api-access-2hgjl\") pod \"limitador-limitador-84994f7ffc-sczjl\" (UID: \"fa87303d-4f34-4bb3-9cf6-9c53882f33eb\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-sczjl"
Sep 18 06:34:57.365687 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.365556    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fa87303d-4f34-4bb3-9cf6-9c53882f33eb-config-file\") pod \"limitador-limitador-84994f7ffc-sczjl\" (UID: \"fa87303d-4f34-4bb3-9cf6-9c53882f33eb\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-sczjl"
Sep 18 06:34:57.366152 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.366133    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fa87303d-4f34-4bb3-9cf6-9c53882f33eb-config-file\") pod \"limitador-limitador-84994f7ffc-sczjl\" (UID: \"fa87303d-4f34-4bb3-9cf6-9c53882f33eb\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-sczjl"
Sep 18 06:34:57.373380 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.373328    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hgjl\" (UniqueName: \"kubernetes.io/projected/fa87303d-4f34-4bb3-9cf6-9c53882f33eb-kube-api-access-2hgjl\") pod \"limitador-limitador-84994f7ffc-sczjl\" (UID: \"fa87303d-4f34-4bb3-9cf6-9c53882f33eb\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-sczjl"
Sep 18 06:34:57.437841 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.437815    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-sczjl"
Sep 18 06:34:57.529243 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.529202    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-z6f7p"]
Sep 18 06:34:57.533767 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.533745    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-z6f7p"
Sep 18 06:34:57.536321 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.536300    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zcv2z\""
Sep 18 06:34:57.539067 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.539015    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-z6f7p"]
Sep 18 06:34:57.564368 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.564337    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:57.632943 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.632888    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:34:57.668255 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.668199    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w99lv\" (UniqueName: \"kubernetes.io/projected/8e7ff514-3fbc-4558-a9fe-e4198843ca3a-kube-api-access-w99lv\") pod \"authorino-7bcfd8f9cd-z6f7p\" (UID: \"8e7ff514-3fbc-4558-a9fe-e4198843ca3a\") " pod="kuadrant-system/authorino-7bcfd8f9cd-z6f7p"
Sep 18 06:34:57.704242 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.704207    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:34:57.706409 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:34:57.706381    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa87303d_4f34_4bb3_9cf6_9c53882f33eb.slice/crio-40c9c7562fcf37c7b5dada0c6062908e10c705a876d3d79e75ee1f036a003728 WatchSource:0}: Error finding container 40c9c7562fcf37c7b5dada0c6062908e10c705a876d3d79e75ee1f036a003728: Status 404 returned error can't find the container with id 40c9c7562fcf37c7b5dada0c6062908e10c705a876d3d79e75ee1f036a003728
Sep 18 06:34:57.769429 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.769407    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w99lv\" (UniqueName: \"kubernetes.io/projected/8e7ff514-3fbc-4558-a9fe-e4198843ca3a-kube-api-access-w99lv\") pod \"authorino-7bcfd8f9cd-z6f7p\" (UID: \"8e7ff514-3fbc-4558-a9fe-e4198843ca3a\") " pod="kuadrant-system/authorino-7bcfd8f9cd-z6f7p"
Sep 18 06:34:57.776967 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.776942    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w99lv\" (UniqueName: \"kubernetes.io/projected/8e7ff514-3fbc-4558-a9fe-e4198843ca3a-kube-api-access-w99lv\") pod \"authorino-7bcfd8f9cd-z6f7p\" (UID: \"8e7ff514-3fbc-4558-a9fe-e4198843ca3a\") " pod="kuadrant-system/authorino-7bcfd8f9cd-z6f7p"
Sep 18 06:34:57.844917 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:57.844892    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-z6f7p"
Sep 18 06:34:58.112440 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:58.112415    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-z6f7p"]
Sep 18 06:34:58.113792 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:34:58.113771    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e7ff514_3fbc_4558_a9fe_e4198843ca3a.slice/crio-97ed256770a00b079fe9ec847c1b190c2da62a0fdf380ab54b54b1096e0c6c5b WatchSource:0}: Error finding container 97ed256770a00b079fe9ec847c1b190c2da62a0fdf380ab54b54b1096e0c6c5b: Status 404 returned error can't find the container with id 97ed256770a00b079fe9ec847c1b190c2da62a0fdf380ab54b54b1096e0c6c5b
Sep 18 06:34:58.574105 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:58.574058    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-sczjl" event={"ID":"fa87303d-4f34-4bb3-9cf6-9c53882f33eb","Type":"ContainerStarted","Data":"40c9c7562fcf37c7b5dada0c6062908e10c705a876d3d79e75ee1f036a003728"}
Sep 18 06:34:58.575375 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:58.575340    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-z6f7p" event={"ID":"8e7ff514-3fbc-4558-a9fe-e4198843ca3a","Type":"ContainerStarted","Data":"97ed256770a00b079fe9ec847c1b190c2da62a0fdf380ab54b54b1096e0c6c5b"}
Sep 18 06:34:59.582192 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:34:59.582155    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-sczjl" event={"ID":"fa87303d-4f34-4bb3-9cf6-9c53882f33eb","Type":"ContainerStarted","Data":"459baa5d943b47172a8dd8d6bb20e7e1a54facdebe05fc5faffa2a900f736879"}
Sep 18 06:35:00.588851 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:00.588814    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-z6f7p" event={"ID":"8e7ff514-3fbc-4558-a9fe-e4198843ca3a","Type":"ContainerStarted","Data":"1c68a9d2284439345412e65780272e279bb77a0d0dd71f26a26490db29e26b18"}
Sep 18 06:35:00.589263 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:00.588948    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-sczjl"
Sep 18 06:35:00.602890 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:00.602845    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-sczjl" podStartSLOduration=2.5957846509999998 podStartE2EDuration="3.60283146s" totalImagesPullingTime="1.007046809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:34:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:34:59.596864724 +0000 UTC m=+765.278392468" watchObservedRunningTime="2026-09-18 06:35:00.60283146 +0000 UTC m=+766.284359204"
Sep 18 06:35:00.604079 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:00.604050    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-z6f7p" podStartSLOduration=1.873514348 podStartE2EDuration="3.604039014s" totalImagesPullingTime="1.730524666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:34:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:35:00.602701253 +0000 UTC m=+766.284228998" watchObservedRunningTime="2026-09-18 06:35:00.604039014 +0000 UTC m=+766.285566797"
Sep 18 06:35:01.879738 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:01.879701    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-z6f7p"]
Sep 18 06:35:02.599943 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:02.599905    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-z6f7p" podUID="8e7ff514-3fbc-4558-a9fe-e4198843ca3a" containerName="authorino" containerID="cri-o://1c68a9d2284439345412e65780272e279bb77a0d0dd71f26a26490db29e26b18" gracePeriod=30
Sep 18 06:35:02.843552 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:02.843529    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-z6f7p"
Sep 18 06:35:02.911628 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:02.911599    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w99lv\" (UniqueName: \"kubernetes.io/projected/8e7ff514-3fbc-4558-a9fe-e4198843ca3a-kube-api-access-w99lv\") pod \"8e7ff514-3fbc-4558-a9fe-e4198843ca3a\" (UID: \"8e7ff514-3fbc-4558-a9fe-e4198843ca3a\") "
Sep 18 06:35:02.913688 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:02.913665    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e7ff514-3fbc-4558-a9fe-e4198843ca3a-kube-api-access-w99lv" (OuterVolumeSpecName: "kube-api-access-w99lv") pod "8e7ff514-3fbc-4558-a9fe-e4198843ca3a" (UID: "8e7ff514-3fbc-4558-a9fe-e4198843ca3a"). InnerVolumeSpecName "kube-api-access-w99lv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:35:03.012409 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:03.012380    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w99lv\" (UniqueName: \"kubernetes.io/projected/8e7ff514-3fbc-4558-a9fe-e4198843ca3a-kube-api-access-w99lv\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:35:03.605830 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:03.605799    2568 generic.go:356] "Generic (PLEG): container finished" podID="8e7ff514-3fbc-4558-a9fe-e4198843ca3a" containerID="1c68a9d2284439345412e65780272e279bb77a0d0dd71f26a26490db29e26b18" exitCode=0
Sep 18 06:35:03.605993 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:03.605844    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-z6f7p"
Sep 18 06:35:03.605993 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:03.605859    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-z6f7p" event={"ID":"8e7ff514-3fbc-4558-a9fe-e4198843ca3a","Type":"ContainerDied","Data":"1c68a9d2284439345412e65780272e279bb77a0d0dd71f26a26490db29e26b18"}
Sep 18 06:35:03.605993 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:03.605890    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-z6f7p" event={"ID":"8e7ff514-3fbc-4558-a9fe-e4198843ca3a","Type":"ContainerDied","Data":"97ed256770a00b079fe9ec847c1b190c2da62a0fdf380ab54b54b1096e0c6c5b"}
Sep 18 06:35:03.605993 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:03.605906    2568 scope.go:117] "RemoveContainer" containerID="1c68a9d2284439345412e65780272e279bb77a0d0dd71f26a26490db29e26b18"
Sep 18 06:35:03.614604 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:03.614589    2568 scope.go:117] "RemoveContainer" containerID="1c68a9d2284439345412e65780272e279bb77a0d0dd71f26a26490db29e26b18"
Sep 18 06:35:03.614872 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:35:03.614847    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c68a9d2284439345412e65780272e279bb77a0d0dd71f26a26490db29e26b18\": container with ID starting with 1c68a9d2284439345412e65780272e279bb77a0d0dd71f26a26490db29e26b18 not found: ID does not exist" containerID="1c68a9d2284439345412e65780272e279bb77a0d0dd71f26a26490db29e26b18"
Sep 18 06:35:03.614967 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:03.614877    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c68a9d2284439345412e65780272e279bb77a0d0dd71f26a26490db29e26b18"} err="failed to get container status \"1c68a9d2284439345412e65780272e279bb77a0d0dd71f26a26490db29e26b18\": rpc error: code = NotFound desc = could not find container \"1c68a9d2284439345412e65780272e279bb77a0d0dd71f26a26490db29e26b18\": container with ID starting with 1c68a9d2284439345412e65780272e279bb77a0d0dd71f26a26490db29e26b18 not found: ID does not exist"
Sep 18 06:35:03.625760 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:03.625736    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-z6f7p"]
Sep 18 06:35:03.627584 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:03.627567    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-z6f7p"]
Sep 18 06:35:04.551909 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:04.551872    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e7ff514-3fbc-4558-a9fe-e4198843ca3a" path="/var/lib/kubelet/pods/8e7ff514-3fbc-4558-a9fe-e4198843ca3a/volumes"
Sep 18 06:35:11.595726 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:11.595698    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-sczjl"
Sep 18 06:35:13.799050 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:13.799014    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:35:13.899213 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:13.899181    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:35:14.222489 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:14.222459    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:35:26.256326 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.256292    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:35:26.387312 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.387280    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:35:26.466967 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.466935    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-pprw8"]
Sep 18 06:35:26.467385 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.467370    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e7ff514-3fbc-4558-a9fe-e4198843ca3a" containerName="authorino"
Sep 18 06:35:26.467441 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.467386    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e7ff514-3fbc-4558-a9fe-e4198843ca3a" containerName="authorino"
Sep 18 06:35:26.467483 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.467462    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e7ff514-3fbc-4558-a9fe-e4198843ca3a" containerName="authorino"
Sep 18 06:35:26.473036 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.473014    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-pprw8"
Sep 18 06:35:26.475568 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.475546    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-zcv2z\""
Sep 18 06:35:26.476811 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.476777    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-pprw8"]
Sep 18 06:35:26.601412 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.601341    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-pprw8"]
Sep 18 06:35:26.601646 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:35:26.601625    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-kszhc], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-pprw8" podUID="de65ea53-a481-4c19-8c34-e208ef501703"
Sep 18 06:35:26.621286 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.621261    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-fqfv5"]
Sep 18 06:35:26.623763 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.623739    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-fqfv5"
Sep 18 06:35:26.624140 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.624111    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kszhc\" (UniqueName: \"kubernetes.io/projected/de65ea53-a481-4c19-8c34-e208ef501703-kube-api-access-kszhc\") pod \"authorino-557f876f69-pprw8\" (UID: \"de65ea53-a481-4c19-8c34-e208ef501703\") " pod="kuadrant-system/authorino-557f876f69-pprw8"
Sep 18 06:35:26.626110 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.626094    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 18 06:35:26.633944 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.633926    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-fqfv5"]
Sep 18 06:35:26.684474 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.684444    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-fqfv5"]
Sep 18 06:35:26.684728 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:35:26.684706    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-rhwq5 tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-fqfv5" podUID="6490eb13-b8ce-4011-ae84-f65703a21e47"
Sep 18 06:35:26.708081 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.708057    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6c887b948-sh45b"]
Sep 18 06:35:26.709516 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.709495    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-pprw8"
Sep 18 06:35:26.709611 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.709495    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-fqfv5"
Sep 18 06:35:26.710373 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.710357    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c887b948-sh45b"
Sep 18 06:35:26.714757 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.714740    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-pprw8"
Sep 18 06:35:26.718082 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.718060    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c887b948-sh45b"]
Sep 18 06:35:26.718371 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.718355    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-fqfv5"
Sep 18 06:35:26.724987 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.724964    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6490eb13-b8ce-4011-ae84-f65703a21e47-tls-cert\") pod \"authorino-6d4dfc649f-fqfv5\" (UID: \"6490eb13-b8ce-4011-ae84-f65703a21e47\") " pod="kuadrant-system/authorino-6d4dfc649f-fqfv5"
Sep 18 06:35:26.725091 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.724997    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhwq5\" (UniqueName: \"kubernetes.io/projected/6490eb13-b8ce-4011-ae84-f65703a21e47-kube-api-access-rhwq5\") pod \"authorino-6d4dfc649f-fqfv5\" (UID: \"6490eb13-b8ce-4011-ae84-f65703a21e47\") " pod="kuadrant-system/authorino-6d4dfc649f-fqfv5"
Sep 18 06:35:26.725164 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.725120    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kszhc\" (UniqueName: \"kubernetes.io/projected/de65ea53-a481-4c19-8c34-e208ef501703-kube-api-access-kszhc\") pod \"authorino-557f876f69-pprw8\" (UID: \"de65ea53-a481-4c19-8c34-e208ef501703\") " pod="kuadrant-system/authorino-557f876f69-pprw8"
Sep 18 06:35:26.732849 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.732825    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kszhc\" (UniqueName: \"kubernetes.io/projected/de65ea53-a481-4c19-8c34-e208ef501703-kube-api-access-kszhc\") pod \"authorino-557f876f69-pprw8\" (UID: \"de65ea53-a481-4c19-8c34-e208ef501703\") " pod="kuadrant-system/authorino-557f876f69-pprw8"
Sep 18 06:35:26.826034 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.826004    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kszhc\" (UniqueName: \"kubernetes.io/projected/de65ea53-a481-4c19-8c34-e208ef501703-kube-api-access-kszhc\") pod \"de65ea53-a481-4c19-8c34-e208ef501703\" (UID: \"de65ea53-a481-4c19-8c34-e208ef501703\") "
Sep 18 06:35:26.826191 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.826114    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b35f8085-93ab-47dd-abde-014893889c99-tls-cert\") pod \"authorino-6c887b948-sh45b\" (UID: \"b35f8085-93ab-47dd-abde-014893889c99\") " pod="kuadrant-system/authorino-6c887b948-sh45b"
Sep 18 06:35:26.826191 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.826137    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckb25\" (UniqueName: \"kubernetes.io/projected/b35f8085-93ab-47dd-abde-014893889c99-kube-api-access-ckb25\") pod \"authorino-6c887b948-sh45b\" (UID: \"b35f8085-93ab-47dd-abde-014893889c99\") " pod="kuadrant-system/authorino-6c887b948-sh45b"
Sep 18 06:35:26.826191 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.826165    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6490eb13-b8ce-4011-ae84-f65703a21e47-tls-cert\") pod \"authorino-6d4dfc649f-fqfv5\" (UID: \"6490eb13-b8ce-4011-ae84-f65703a21e47\") " pod="kuadrant-system/authorino-6d4dfc649f-fqfv5"
Sep 18 06:35:26.826191 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.826184    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhwq5\" (UniqueName: \"kubernetes.io/projected/6490eb13-b8ce-4011-ae84-f65703a21e47-kube-api-access-rhwq5\") pod \"authorino-6d4dfc649f-fqfv5\" (UID: \"6490eb13-b8ce-4011-ae84-f65703a21e47\") " pod="kuadrant-system/authorino-6d4dfc649f-fqfv5"
Sep 18 06:35:26.828176 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.828145    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de65ea53-a481-4c19-8c34-e208ef501703-kube-api-access-kszhc" (OuterVolumeSpecName: "kube-api-access-kszhc") pod "de65ea53-a481-4c19-8c34-e208ef501703" (UID: "de65ea53-a481-4c19-8c34-e208ef501703"). InnerVolumeSpecName "kube-api-access-kszhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:35:26.828536 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.828520    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6490eb13-b8ce-4011-ae84-f65703a21e47-tls-cert\") pod \"authorino-6d4dfc649f-fqfv5\" (UID: \"6490eb13-b8ce-4011-ae84-f65703a21e47\") " pod="kuadrant-system/authorino-6d4dfc649f-fqfv5"
Sep 18 06:35:26.834426 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.834403    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhwq5\" (UniqueName: \"kubernetes.io/projected/6490eb13-b8ce-4011-ae84-f65703a21e47-kube-api-access-rhwq5\") pod \"authorino-6d4dfc649f-fqfv5\" (UID: \"6490eb13-b8ce-4011-ae84-f65703a21e47\") " pod="kuadrant-system/authorino-6d4dfc649f-fqfv5"
Sep 18 06:35:26.926964 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.926945    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6490eb13-b8ce-4011-ae84-f65703a21e47-tls-cert\") pod \"6490eb13-b8ce-4011-ae84-f65703a21e47\" (UID: \"6490eb13-b8ce-4011-ae84-f65703a21e47\") "
Sep 18 06:35:26.927086 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.927015    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhwq5\" (UniqueName: \"kubernetes.io/projected/6490eb13-b8ce-4011-ae84-f65703a21e47-kube-api-access-rhwq5\") pod \"6490eb13-b8ce-4011-ae84-f65703a21e47\" (UID: \"6490eb13-b8ce-4011-ae84-f65703a21e47\") "
Sep 18 06:35:26.927137 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.927124    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b35f8085-93ab-47dd-abde-014893889c99-tls-cert\") pod \"authorino-6c887b948-sh45b\" (UID: \"b35f8085-93ab-47dd-abde-014893889c99\") " pod="kuadrant-system/authorino-6c887b948-sh45b"
Sep 18 06:35:26.927175 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.927150    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckb25\" (UniqueName: \"kubernetes.io/projected/b35f8085-93ab-47dd-abde-014893889c99-kube-api-access-ckb25\") pod \"authorino-6c887b948-sh45b\" (UID: \"b35f8085-93ab-47dd-abde-014893889c99\") " pod="kuadrant-system/authorino-6c887b948-sh45b"
Sep 18 06:35:26.927299 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.927285    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kszhc\" (UniqueName: \"kubernetes.io/projected/de65ea53-a481-4c19-8c34-e208ef501703-kube-api-access-kszhc\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:35:26.928903 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.928872    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6490eb13-b8ce-4011-ae84-f65703a21e47-kube-api-access-rhwq5" (OuterVolumeSpecName: "kube-api-access-rhwq5") pod "6490eb13-b8ce-4011-ae84-f65703a21e47" (UID: "6490eb13-b8ce-4011-ae84-f65703a21e47"). InnerVolumeSpecName "kube-api-access-rhwq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:35:26.928998 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.928940    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6490eb13-b8ce-4011-ae84-f65703a21e47-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "6490eb13-b8ce-4011-ae84-f65703a21e47" (UID: "6490eb13-b8ce-4011-ae84-f65703a21e47"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:35:26.929401 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.929383    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b35f8085-93ab-47dd-abde-014893889c99-tls-cert\") pod \"authorino-6c887b948-sh45b\" (UID: \"b35f8085-93ab-47dd-abde-014893889c99\") " pod="kuadrant-system/authorino-6c887b948-sh45b"
Sep 18 06:35:26.934445 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:26.934429    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckb25\" (UniqueName: \"kubernetes.io/projected/b35f8085-93ab-47dd-abde-014893889c99-kube-api-access-ckb25\") pod \"authorino-6c887b948-sh45b\" (UID: \"b35f8085-93ab-47dd-abde-014893889c99\") " pod="kuadrant-system/authorino-6c887b948-sh45b"
Sep 18 06:35:27.026381 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:27.026347    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c887b948-sh45b"
Sep 18 06:35:27.027708 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:27.027687    2568 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6490eb13-b8ce-4011-ae84-f65703a21e47-tls-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:35:27.027806 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:27.027711    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhwq5\" (UniqueName: \"kubernetes.io/projected/6490eb13-b8ce-4011-ae84-f65703a21e47-kube-api-access-rhwq5\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:35:27.290984 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:27.290957    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c887b948-sh45b"]
Sep 18 06:35:27.292462 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:35:27.292437    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb35f8085_93ab_47dd_abde_014893889c99.slice/crio-c95bce9bf42bf9069d2581b6e830a495449eb32164e67b8541d6a5084d2231a7 WatchSource:0}: Error finding container c95bce9bf42bf9069d2581b6e830a495449eb32164e67b8541d6a5084d2231a7: Status 404 returned error can't find the container with id c95bce9bf42bf9069d2581b6e830a495449eb32164e67b8541d6a5084d2231a7
Sep 18 06:35:27.716105 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:27.716067    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c887b948-sh45b" event={"ID":"b35f8085-93ab-47dd-abde-014893889c99","Type":"ContainerStarted","Data":"c95bce9bf42bf9069d2581b6e830a495449eb32164e67b8541d6a5084d2231a7"}
Sep 18 06:35:27.716105 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:27.716100    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-pprw8"
Sep 18 06:35:27.716325 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:27.716079    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-fqfv5"
Sep 18 06:35:27.744586 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:27.744554    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-pprw8"]
Sep 18 06:35:27.746490 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:27.746469    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-pprw8"]
Sep 18 06:35:27.764875 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:27.764855    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-fqfv5"]
Sep 18 06:35:27.767898 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:27.767876    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-fqfv5"]
Sep 18 06:35:28.542816 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:28.542783    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6490eb13-b8ce-4011-ae84-f65703a21e47" path="/var/lib/kubelet/pods/6490eb13-b8ce-4011-ae84-f65703a21e47/volumes"
Sep 18 06:35:28.543183 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:28.543002    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de65ea53-a481-4c19-8c34-e208ef501703" path="/var/lib/kubelet/pods/de65ea53-a481-4c19-8c34-e208ef501703/volumes"
Sep 18 06:35:28.722296 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:28.722260    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c887b948-sh45b" event={"ID":"b35f8085-93ab-47dd-abde-014893889c99","Type":"ContainerStarted","Data":"86a2d66cd04233798f9945271bb76b07131eaeb9240c391a7fe864792c2edd27"}
Sep 18 06:35:28.740756 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:28.740711    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6c887b948-sh45b" podStartSLOduration=2.165269407 podStartE2EDuration="2.740696902s" totalImagesPullingTime="575.427495ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:35:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:35:28.737990877 +0000 UTC m=+794.419518637" watchObservedRunningTime="2026-09-18 06:35:28.740696902 +0000 UTC m=+794.422224646"
Sep 18 06:35:33.224512 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:33.224476    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:35:33.322236 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:33.322186    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:35:33.407297 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:33.407270    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:35:33.460732 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:33.460696    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:35:33.560929 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:33.560855    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:35:36.663621 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:36.663589    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:35:36.764477 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:36.764447    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:35:36.878132 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:36.878104    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:35:52.842577 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:52.842542    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:35:52.943516 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:52.943490    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:35:53.614485 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:35:53.614450    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:36:08.243955 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:08.243918    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:36:08.346236 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:08.346183    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:36:09.654655 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:09.654626    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:36:09.769055 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:09.769024    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:36:09.859264 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:09.859232    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:36:10.019205 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:10.019129    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:36:10.300243 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:10.300149    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:36:10.392874 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:10.392827    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:36:10.438685 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:10.438648    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:36:18.561352 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:18.561318    2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-f2pj4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 06:36:18.561751 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:18.561385    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4" podUID="0484017e-ee10-41d4-b810-b3a80f6382da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 06:36:19.965958 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:19.965927    2568 generic.go:356] "Generic (PLEG): container finished" podID="0484017e-ee10-41d4-b810-b3a80f6382da" containerID="6761e3acbfb5c7c7c74bc9c6769482a8985f0a34ef7651aee9ce7ab49124cc7d" exitCode=0
Sep 18 06:36:19.966404 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:19.965998    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4" event={"ID":"0484017e-ee10-41d4-b810-b3a80f6382da","Type":"ContainerDied","Data":"6761e3acbfb5c7c7c74bc9c6769482a8985f0a34ef7651aee9ce7ab49124cc7d"}
Sep 18 06:36:19.966404 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:19.966394    2568 scope.go:117] "RemoveContainer" containerID="6761e3acbfb5c7c7c74bc9c6769482a8985f0a34ef7651aee9ce7ab49124cc7d"
Sep 18 06:36:19.967469 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:19.967443    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:36:19.967544 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:19.967508    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:36:19.967544 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:19.967540    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:36:20.972892 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:20.972856    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4" event={"ID":"0484017e-ee10-41d4-b810-b3a80f6382da","Type":"ContainerStarted","Data":"f935360312ad2ae1daef9b74d6eb5a0cfd03bfc44a333cef9c3455380987c2d4"}
Sep 18 06:36:21.146256 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:21.146032    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:36:21.245701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:21.245498    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:36:21.637761 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:21.637665    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:36:21.637949 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:21.637841    2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-f2pj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 18 06:36:21.637949 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:21.637895    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4" podUID="0484017e-ee10-41d4-b810-b3a80f6382da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 18 06:36:22.638133 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:22.638095    2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-f2pj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 18 06:36:22.638653 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:22.638165    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4" podUID="0484017e-ee10-41d4-b810-b3a80f6382da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 18 06:36:23.638002 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:23.637963    2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-f2pj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 18 06:36:23.638358 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:23.638323    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4" podUID="0484017e-ee10-41d4-b810-b3a80f6382da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 18 06:36:24.638828 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:24.638400    2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-f2pj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 18 06:36:24.638828 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:24.638461    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4" podUID="0484017e-ee10-41d4-b810-b3a80f6382da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 18 06:36:25.637657 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:25.637613    2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-f2pj4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body=
Sep 18 06:36:25.637965 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:25.637932    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4" podUID="0484017e-ee10-41d4-b810-b3a80f6382da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused"
Sep 18 06:36:26.641546 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:26.641518    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:36:26.641913 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:26.641791    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:36:26.642553 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:26.642531    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4"
Sep 18 06:36:35.280651 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:35.280614    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:36:35.681832 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:36:35.681792    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:37:10.714058 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:10.714023    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:37:11.480411 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:11.480377    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:37:30.280665 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:30.280630    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:37:31.178912 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:31.178871    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:37:36.479635 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:36.479603    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:37:37.683777 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:37.683741    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:37:47.780656 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.780622    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"]
Sep 18 06:37:47.783340 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.783316    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:47.785917 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.785893    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 18 06:37:47.786886 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.786869    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wzjhw\""
Sep 18 06:37:47.787096 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.787081    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 18 06:37:47.787198 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.787101    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 18 06:37:47.793538 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.793512    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"]
Sep 18 06:37:47.809828 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.809798    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0111315-8eb7-45db-8729-e99cb59b4df6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88\" (UID: \"b0111315-8eb7-45db-8729-e99cb59b4df6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:47.809937 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.809908    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0111315-8eb7-45db-8729-e99cb59b4df6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88\" (UID: \"b0111315-8eb7-45db-8729-e99cb59b4df6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:47.809984 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.809951    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tshk\" (UniqueName: \"kubernetes.io/projected/b0111315-8eb7-45db-8729-e99cb59b4df6-kube-api-access-8tshk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88\" (UID: \"b0111315-8eb7-45db-8729-e99cb59b4df6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:47.810027 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.809985    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0111315-8eb7-45db-8729-e99cb59b4df6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88\" (UID: \"b0111315-8eb7-45db-8729-e99cb59b4df6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:47.810066 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.810032    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0111315-8eb7-45db-8729-e99cb59b4df6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88\" (UID: \"b0111315-8eb7-45db-8729-e99cb59b4df6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:47.810113 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.810072    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0111315-8eb7-45db-8729-e99cb59b4df6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88\" (UID: \"b0111315-8eb7-45db-8729-e99cb59b4df6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:47.819833 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.819765    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:37:47.911141 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.911106    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0111315-8eb7-45db-8729-e99cb59b4df6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88\" (UID: \"b0111315-8eb7-45db-8729-e99cb59b4df6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:47.911367 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.911158    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tshk\" (UniqueName: \"kubernetes.io/projected/b0111315-8eb7-45db-8729-e99cb59b4df6-kube-api-access-8tshk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88\" (UID: \"b0111315-8eb7-45db-8729-e99cb59b4df6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:47.911367 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.911191    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0111315-8eb7-45db-8729-e99cb59b4df6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88\" (UID: \"b0111315-8eb7-45db-8729-e99cb59b4df6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:47.911367 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.911238    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0111315-8eb7-45db-8729-e99cb59b4df6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88\" (UID: \"b0111315-8eb7-45db-8729-e99cb59b4df6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:47.911367 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.911265    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0111315-8eb7-45db-8729-e99cb59b4df6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88\" (UID: \"b0111315-8eb7-45db-8729-e99cb59b4df6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:47.911367 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.911313    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0111315-8eb7-45db-8729-e99cb59b4df6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88\" (UID: \"b0111315-8eb7-45db-8729-e99cb59b4df6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:47.911655 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.911598    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0111315-8eb7-45db-8729-e99cb59b4df6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88\" (UID: \"b0111315-8eb7-45db-8729-e99cb59b4df6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:47.911655 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.911623    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0111315-8eb7-45db-8729-e99cb59b4df6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88\" (UID: \"b0111315-8eb7-45db-8729-e99cb59b4df6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:47.911759 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.911667    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0111315-8eb7-45db-8729-e99cb59b4df6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88\" (UID: \"b0111315-8eb7-45db-8729-e99cb59b4df6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:47.913542 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.913523    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0111315-8eb7-45db-8729-e99cb59b4df6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88\" (UID: \"b0111315-8eb7-45db-8729-e99cb59b4df6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:47.913715 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.913698    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0111315-8eb7-45db-8729-e99cb59b4df6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88\" (UID: \"b0111315-8eb7-45db-8729-e99cb59b4df6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:47.918891 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:47.918857    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tshk\" (UniqueName: \"kubernetes.io/projected/b0111315-8eb7-45db-8729-e99cb59b4df6-kube-api-access-8tshk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88\" (UID: \"b0111315-8eb7-45db-8729-e99cb59b4df6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:48.096011 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:48.095927    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:48.365415 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:48.365390    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"]
Sep 18 06:37:48.367390 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:37:48.367358    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0111315_8eb7_45db_8729_e99cb59b4df6.slice/crio-047780703d92193b26ff3b831d206e2548403cf42eca19ca4c2939249664718a WatchSource:0}: Error finding container 047780703d92193b26ff3b831d206e2548403cf42eca19ca4c2939249664718a: Status 404 returned error can't find the container with id 047780703d92193b26ff3b831d206e2548403cf42eca19ca4c2939249664718a
Sep 18 06:37:48.369099 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:48.369081    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 06:37:48.376235 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:48.376192    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88" event={"ID":"b0111315-8eb7-45db-8729-e99cb59b4df6","Type":"ContainerStarted","Data":"047780703d92193b26ff3b831d206e2548403cf42eca19ca4c2939249664718a"}
Sep 18 06:37:49.382513 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:49.382484    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88" event={"ID":"b0111315-8eb7-45db-8729-e99cb59b4df6","Type":"ContainerStarted","Data":"2251b23648dc5eefed9a2973d28a7b18ea39106d2c8d540e31aa00b1084ec6e7"}
Sep 18 06:37:49.398555 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:49.398503    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88" podStartSLOduration=1.473971112 podStartE2EDuration="2.398487356s" totalImagesPullingTime="924.516244ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:37:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:37:49.397546001 +0000 UTC m=+935.079073745" watchObservedRunningTime="2026-09-18 06:37:49.398487356 +0000 UTC m=+935.080015100"
Sep 18 06:37:49.586349 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:49.586271    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:37:58.096758 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:58.096719    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:58.097191 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:58.096801    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:58.109104 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:58.109079    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:37:58.435998 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:37:58.435968    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88"
Sep 18 06:38:15.991597 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:15.991559    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"]
Sep 18 06:38:15.995256 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:15.995208    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:15.997552 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:15.997531    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 18 06:38:16.004615 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:16.004593    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"]
Sep 18 06:38:16.018540 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:16.018508    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:38:16.052775 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:16.052746    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8868048f-b3d5-4257-97ca-027cd06d1fc2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5j6g\" (UID: \"8868048f-b3d5-4257-97ca-027cd06d1fc2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:16.052934 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:16.052811    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8868048f-b3d5-4257-97ca-027cd06d1fc2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5j6g\" (UID: \"8868048f-b3d5-4257-97ca-027cd06d1fc2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:16.052934 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:16.052830    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8868048f-b3d5-4257-97ca-027cd06d1fc2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5j6g\" (UID: \"8868048f-b3d5-4257-97ca-027cd06d1fc2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:16.052934 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:16.052858    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55t4l\" (UniqueName: \"kubernetes.io/projected/8868048f-b3d5-4257-97ca-027cd06d1fc2-kube-api-access-55t4l\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5j6g\" (UID: \"8868048f-b3d5-4257-97ca-027cd06d1fc2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:16.053097 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:16.052974    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8868048f-b3d5-4257-97ca-027cd06d1fc2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5j6g\" (UID: \"8868048f-b3d5-4257-97ca-027cd06d1fc2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:16.053097 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:16.052996    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8868048f-b3d5-4257-97ca-027cd06d1fc2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5j6g\" (UID: \"8868048f-b3d5-4257-97ca-027cd06d1fc2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:16.154004 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:16.153967    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8868048f-b3d5-4257-97ca-027cd06d1fc2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5j6g\" (UID: \"8868048f-b3d5-4257-97ca-027cd06d1fc2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:16.154004 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:16.154001    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8868048f-b3d5-4257-97ca-027cd06d1fc2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5j6g\" (UID: \"8868048f-b3d5-4257-97ca-027cd06d1fc2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:16.154307 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:16.154034    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55t4l\" (UniqueName: \"kubernetes.io/projected/8868048f-b3d5-4257-97ca-027cd06d1fc2-kube-api-access-55t4l\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5j6g\" (UID: \"8868048f-b3d5-4257-97ca-027cd06d1fc2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:16.154307 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:16.154158    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8868048f-b3d5-4257-97ca-027cd06d1fc2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5j6g\" (UID: \"8868048f-b3d5-4257-97ca-027cd06d1fc2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:16.154307 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:16.154199    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8868048f-b3d5-4257-97ca-027cd06d1fc2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5j6g\" (UID: \"8868048f-b3d5-4257-97ca-027cd06d1fc2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:16.154307 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:16.154291    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8868048f-b3d5-4257-97ca-027cd06d1fc2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5j6g\" (UID: \"8868048f-b3d5-4257-97ca-027cd06d1fc2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:16.154580 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:16.154557    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8868048f-b3d5-4257-97ca-027cd06d1fc2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5j6g\" (UID: \"8868048f-b3d5-4257-97ca-027cd06d1fc2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:16.154697 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:16.154590    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8868048f-b3d5-4257-97ca-027cd06d1fc2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5j6g\" (UID: \"8868048f-b3d5-4257-97ca-027cd06d1fc2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:16.154697 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:16.154643    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8868048f-b3d5-4257-97ca-027cd06d1fc2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5j6g\" (UID: \"8868048f-b3d5-4257-97ca-027cd06d1fc2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:16.156372 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:16.156353    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8868048f-b3d5-4257-97ca-027cd06d1fc2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5j6g\" (UID: \"8868048f-b3d5-4257-97ca-027cd06d1fc2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:16.156541 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:16.156524    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8868048f-b3d5-4257-97ca-027cd06d1fc2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5j6g\" (UID: \"8868048f-b3d5-4257-97ca-027cd06d1fc2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:16.161390 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:16.161358    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55t4l\" (UniqueName: \"kubernetes.io/projected/8868048f-b3d5-4257-97ca-027cd06d1fc2-kube-api-access-55t4l\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-h5j6g\" (UID: \"8868048f-b3d5-4257-97ca-027cd06d1fc2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:16.306405 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:16.306319    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:16.575627 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:16.575558    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"]
Sep 18 06:38:16.578474 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:38:16.578448    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8868048f_b3d5_4257_97ca_027cd06d1fc2.slice/crio-e8ef89386c865d91d4e3df02f168e510cf8b8e804034ec742b279760140f26de WatchSource:0}: Error finding container e8ef89386c865d91d4e3df02f168e510cf8b8e804034ec742b279760140f26de: Status 404 returned error can't find the container with id e8ef89386c865d91d4e3df02f168e510cf8b8e804034ec742b279760140f26de
Sep 18 06:38:17.197335 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:17.197294    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:38:17.513539 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:17.513456    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g" event={"ID":"8868048f-b3d5-4257-97ca-027cd06d1fc2","Type":"ContainerStarted","Data":"2528c024d56cbf7a09071fd5e97256599f8100df1f6ce97172d0f023824695a6"}
Sep 18 06:38:17.513539 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:17.513508    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g" event={"ID":"8868048f-b3d5-4257-97ca-027cd06d1fc2","Type":"ContainerStarted","Data":"e8ef89386c865d91d4e3df02f168e510cf8b8e804034ec742b279760140f26de"}
Sep 18 06:38:17.530685 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:17.530634    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g" podStartSLOduration=2.360205587 podStartE2EDuration="2.530616859s" totalImagesPullingTime="170.411272ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:38:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:38:17.528774168 +0000 UTC m=+963.210301916" watchObservedRunningTime="2026-09-18 06:38:17.530616859 +0000 UTC m=+963.212144604"
Sep 18 06:38:26.306647 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:26.306608    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:26.307261 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:26.306701    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:26.318808 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:26.318783    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:26.570585 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:26.570500    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-h5j6g"
Sep 18 06:38:36.573687 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.573593    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"]
Sep 18 06:38:36.577838 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.577817    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:36.580566 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.580541    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 18 06:38:36.586829 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.586805    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"]
Sep 18 06:38:36.610112 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.610082    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:38:36.635777 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.635741    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvwhp\" (UniqueName: \"kubernetes.io/projected/d0ef703e-fddd-4851-956f-ab20938918eb-kube-api-access-jvwhp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dr59q\" (UID: \"d0ef703e-fddd-4851-956f-ab20938918eb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:36.635916 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.635791    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ef703e-fddd-4851-956f-ab20938918eb-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dr59q\" (UID: \"d0ef703e-fddd-4851-956f-ab20938918eb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:36.635916 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.635816    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ef703e-fddd-4851-956f-ab20938918eb-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dr59q\" (UID: \"d0ef703e-fddd-4851-956f-ab20938918eb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:36.635916 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.635833    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ef703e-fddd-4851-956f-ab20938918eb-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dr59q\" (UID: \"d0ef703e-fddd-4851-956f-ab20938918eb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:36.635916 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.635906    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0ef703e-fddd-4851-956f-ab20938918eb-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dr59q\" (UID: \"d0ef703e-fddd-4851-956f-ab20938918eb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:36.636059 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.635957    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ef703e-fddd-4851-956f-ab20938918eb-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dr59q\" (UID: \"d0ef703e-fddd-4851-956f-ab20938918eb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:36.737098 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.737066    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ef703e-fddd-4851-956f-ab20938918eb-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dr59q\" (UID: \"d0ef703e-fddd-4851-956f-ab20938918eb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:36.737098 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.737108    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ef703e-fddd-4851-956f-ab20938918eb-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dr59q\" (UID: \"d0ef703e-fddd-4851-956f-ab20938918eb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:36.737383 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.737126    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ef703e-fddd-4851-956f-ab20938918eb-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dr59q\" (UID: \"d0ef703e-fddd-4851-956f-ab20938918eb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:36.737383 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.737173    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0ef703e-fddd-4851-956f-ab20938918eb-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dr59q\" (UID: \"d0ef703e-fddd-4851-956f-ab20938918eb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:36.737383 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.737193    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ef703e-fddd-4851-956f-ab20938918eb-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dr59q\" (UID: \"d0ef703e-fddd-4851-956f-ab20938918eb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:36.737383 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.737301    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvwhp\" (UniqueName: \"kubernetes.io/projected/d0ef703e-fddd-4851-956f-ab20938918eb-kube-api-access-jvwhp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dr59q\" (UID: \"d0ef703e-fddd-4851-956f-ab20938918eb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:36.737639 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.737614    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0ef703e-fddd-4851-956f-ab20938918eb-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dr59q\" (UID: \"d0ef703e-fddd-4851-956f-ab20938918eb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:36.737697 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.737654    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0ef703e-fddd-4851-956f-ab20938918eb-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dr59q\" (UID: \"d0ef703e-fddd-4851-956f-ab20938918eb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:36.737901 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.737877    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0ef703e-fddd-4851-956f-ab20938918eb-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dr59q\" (UID: \"d0ef703e-fddd-4851-956f-ab20938918eb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:36.739376 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.739358    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0ef703e-fddd-4851-956f-ab20938918eb-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dr59q\" (UID: \"d0ef703e-fddd-4851-956f-ab20938918eb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:36.739703 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.739685    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0ef703e-fddd-4851-956f-ab20938918eb-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dr59q\" (UID: \"d0ef703e-fddd-4851-956f-ab20938918eb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:36.744987 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.744957    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvwhp\" (UniqueName: \"kubernetes.io/projected/d0ef703e-fddd-4851-956f-ab20938918eb-kube-api-access-jvwhp\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-dr59q\" (UID: \"d0ef703e-fddd-4851-956f-ab20938918eb\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:36.890629 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:36.890541    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:37.160327 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:37.160301    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"]
Sep 18 06:38:37.162813 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:38:37.162783    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0ef703e_fddd_4851_956f_ab20938918eb.slice/crio-32f3e3b416ed4b7025f48d1b17b5bc1c62b91b4cf2726ad80c919a3a2c16078f WatchSource:0}: Error finding container 32f3e3b416ed4b7025f48d1b17b5bc1c62b91b4cf2726ad80c919a3a2c16078f: Status 404 returned error can't find the container with id 32f3e3b416ed4b7025f48d1b17b5bc1c62b91b4cf2726ad80c919a3a2c16078f
Sep 18 06:38:37.613046 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:37.612961    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q" event={"ID":"d0ef703e-fddd-4851-956f-ab20938918eb","Type":"ContainerStarted","Data":"32f3e3b416ed4b7025f48d1b17b5bc1c62b91b4cf2726ad80c919a3a2c16078f"}
Sep 18 06:38:38.620164 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:38.620127    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q" event={"ID":"d0ef703e-fddd-4851-956f-ab20938918eb","Type":"ContainerStarted","Data":"2885003e4fa630180c74534c87aa9d7a5011fbed98d8ef7de2c493b1b3a668d5"}
Sep 18 06:38:38.638096 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:38.638032    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q" podStartSLOduration=2.187066136 podStartE2EDuration="2.638012478s" totalImagesPullingTime="450.946342ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:38:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:38:38.635731625 +0000 UTC m=+984.317259370" watchObservedRunningTime="2026-09-18 06:38:38.638012478 +0000 UTC m=+984.319540225"
Sep 18 06:38:38.885705 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:38.885623    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:38:46.890836 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:46.890801    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:46.890836 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:46.890844    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:46.903266 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:46.903244    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:38:47.673729 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:38:47.673696    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-dr59q"
Sep 18 06:40:06.273628 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:06.273545    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c887b948-sh45b"]
Sep 18 06:40:06.274154 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:06.273773    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6c887b948-sh45b" podUID="b35f8085-93ab-47dd-abde-014893889c99" containerName="authorino" containerID="cri-o://86a2d66cd04233798f9945271bb76b07131eaeb9240c391a7fe864792c2edd27" gracePeriod=30
Sep 18 06:40:06.527800 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:06.527744    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c887b948-sh45b"
Sep 18 06:40:06.551259 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:06.551211    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b35f8085-93ab-47dd-abde-014893889c99-tls-cert\") pod \"b35f8085-93ab-47dd-abde-014893889c99\" (UID: \"b35f8085-93ab-47dd-abde-014893889c99\") "
Sep 18 06:40:06.551434 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:06.551308    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ckb25\" (UniqueName: \"kubernetes.io/projected/b35f8085-93ab-47dd-abde-014893889c99-kube-api-access-ckb25\") pod \"b35f8085-93ab-47dd-abde-014893889c99\" (UID: \"b35f8085-93ab-47dd-abde-014893889c99\") "
Sep 18 06:40:06.553241 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:06.553187    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b35f8085-93ab-47dd-abde-014893889c99-kube-api-access-ckb25" (OuterVolumeSpecName: "kube-api-access-ckb25") pod "b35f8085-93ab-47dd-abde-014893889c99" (UID: "b35f8085-93ab-47dd-abde-014893889c99"). InnerVolumeSpecName "kube-api-access-ckb25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:40:06.563093 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:06.563060    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b35f8085-93ab-47dd-abde-014893889c99-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "b35f8085-93ab-47dd-abde-014893889c99" (UID: "b35f8085-93ab-47dd-abde-014893889c99"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:40:06.652292 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:06.652263    2568 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b35f8085-93ab-47dd-abde-014893889c99-tls-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:40:06.652292 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:06.652286    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ckb25\" (UniqueName: \"kubernetes.io/projected/b35f8085-93ab-47dd-abde-014893889c99-kube-api-access-ckb25\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:40:06.805276 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:06.805171    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:06.918160 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:06.918131    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:07.035068 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:07.035036    2568 generic.go:356] "Generic (PLEG): container finished" podID="b35f8085-93ab-47dd-abde-014893889c99" containerID="86a2d66cd04233798f9945271bb76b07131eaeb9240c391a7fe864792c2edd27" exitCode=0
Sep 18 06:40:07.035272 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:07.035081    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c887b948-sh45b"
Sep 18 06:40:07.035272 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:07.035110    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c887b948-sh45b" event={"ID":"b35f8085-93ab-47dd-abde-014893889c99","Type":"ContainerDied","Data":"86a2d66cd04233798f9945271bb76b07131eaeb9240c391a7fe864792c2edd27"}
Sep 18 06:40:07.035272 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:07.035156    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c887b948-sh45b" event={"ID":"b35f8085-93ab-47dd-abde-014893889c99","Type":"ContainerDied","Data":"c95bce9bf42bf9069d2581b6e830a495449eb32164e67b8541d6a5084d2231a7"}
Sep 18 06:40:07.035272 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:07.035179    2568 scope.go:117] "RemoveContainer" containerID="86a2d66cd04233798f9945271bb76b07131eaeb9240c391a7fe864792c2edd27"
Sep 18 06:40:07.044889 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:07.044871    2568 scope.go:117] "RemoveContainer" containerID="86a2d66cd04233798f9945271bb76b07131eaeb9240c391a7fe864792c2edd27"
Sep 18 06:40:07.045192 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:40:07.045172    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86a2d66cd04233798f9945271bb76b07131eaeb9240c391a7fe864792c2edd27\": container with ID starting with 86a2d66cd04233798f9945271bb76b07131eaeb9240c391a7fe864792c2edd27 not found: ID does not exist" containerID="86a2d66cd04233798f9945271bb76b07131eaeb9240c391a7fe864792c2edd27"
Sep 18 06:40:07.045325 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:07.045197    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86a2d66cd04233798f9945271bb76b07131eaeb9240c391a7fe864792c2edd27"} err="failed to get container status \"86a2d66cd04233798f9945271bb76b07131eaeb9240c391a7fe864792c2edd27\": rpc error: code = NotFound desc = could not find container \"86a2d66cd04233798f9945271bb76b07131eaeb9240c391a7fe864792c2edd27\": container with ID starting with 86a2d66cd04233798f9945271bb76b07131eaeb9240c391a7fe864792c2edd27 not found: ID does not exist"
Sep 18 06:40:07.055749 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:07.055681    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c887b948-sh45b"]
Sep 18 06:40:07.058815 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:07.058793    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6c887b948-sh45b"]
Sep 18 06:40:07.804711 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:07.804679    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:07.904809 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:07.904778    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:08.543950 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:08.543916    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b35f8085-93ab-47dd-abde-014893889c99" path="/var/lib/kubelet/pods/b35f8085-93ab-47dd-abde-014893889c99/volumes"
Sep 18 06:40:23.029264 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.029205    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:23.083421 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.083388    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:23.145755 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.145729    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:23.335539 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.335454    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:23.403008 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.402977    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:23.449864 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.449827    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"]
Sep 18 06:40:23.451133 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.451111    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b35f8085-93ab-47dd-abde-014893889c99" containerName="authorino"
Sep 18 06:40:23.451225 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.451137    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b35f8085-93ab-47dd-abde-014893889c99" containerName="authorino"
Sep 18 06:40:23.452166 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.451965    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b35f8085-93ab-47dd-abde-014893889c99" containerName="authorino"
Sep 18 06:40:23.465516 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.465484    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.466019 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.465995    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"]
Sep 18 06:40:23.468442 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.468410    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-946609-openshift-default-dockercfg-6n2mw\""
Sep 18 06:40:23.476591 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.476565    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:23.579048 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.579020    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:23.604272 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.604178    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-workload-socket\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.604272 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.604211    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szrfs\" (UniqueName: \"kubernetes.io/projected/d88eebd0-4865-4103-ab85-92c6c00cbd74-kube-api-access-szrfs\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.604435 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.604303    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-envoy\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.604435 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.604374    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d88eebd0-4865-4103-ab85-92c6c00cbd74-istiod-ca-cert\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.604513 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.604466    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-data\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.604513 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.604506    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-workload-certs\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.604628 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.604580    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-podinfo\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.604667 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.604649    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-credential-socket\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.604705 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.604680    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-token\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.676919 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.676896    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:23.705521 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.705496    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-data\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.705617 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.705527    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-workload-certs\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.705617 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.705550    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-podinfo\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.705617 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.705570    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-credential-socket\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.705617 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.705587    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-token\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.705821 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.705626    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-workload-socket\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.705821 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.705773    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szrfs\" (UniqueName: \"kubernetes.io/projected/d88eebd0-4865-4103-ab85-92c6c00cbd74-kube-api-access-szrfs\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.705925 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.705825    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-envoy\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.705925 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.705898    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d88eebd0-4865-4103-ab85-92c6c00cbd74-istiod-ca-cert\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.706041 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.705926    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-workload-certs\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.706041 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.706028    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-data\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.706129 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.706041    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-workload-socket\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.706129 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.706075    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-credential-socket\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.706469 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.706448    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d88eebd0-4865-4103-ab85-92c6c00cbd74-istiod-ca-cert\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.707937 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.707912    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-podinfo\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.708013 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.707919    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-envoy\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.713141 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.713122    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-token\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.713323 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.713303    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szrfs\" (UniqueName: \"kubernetes.io/projected/d88eebd0-4865-4103-ab85-92c6c00cbd74-kube-api-access-szrfs\") pod \"e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") " pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:23.777494 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.777461    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:23.779517 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:23.779496    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:24.056106 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:24.056080    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"]
Sep 18 06:40:24.058285 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:40:24.058258    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd88eebd0_4865_4103_ab85_92c6c00cbd74.slice/crio-e47ae867012b597ef2349d9dac4da6c5f0702db047a84b3cf5a03fe647fbac81 WatchSource:0}: Error finding container e47ae867012b597ef2349d9dac4da6c5f0702db047a84b3cf5a03fe647fbac81: Status 404 returned error can't find the container with id e47ae867012b597ef2349d9dac4da6c5f0702db047a84b3cf5a03fe647fbac81
Sep 18 06:40:24.060687 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:24.060657    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:40:24.060764 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:24.060729    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:40:24.060764 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:24.060759    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:40:24.114911 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:24.114885    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b" event={"ID":"d88eebd0-4865-4103-ab85-92c6c00cbd74","Type":"ContainerStarted","Data":"e47ae867012b597ef2349d9dac4da6c5f0702db047a84b3cf5a03fe647fbac81"}
Sep 18 06:40:24.448074 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:24.448044    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:24.558034 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:24.558005    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:24.904090 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:24.904058    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:24.968361 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:24.968330    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7765-d25e23"]
Sep 18 06:40:24.971962 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:24.971945    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7765-d25e23"
Sep 18 06:40:24.974266 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:24.974210    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-h78xn\""
Sep 18 06:40:24.978253 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:24.978232    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7765-d25e23"]
Sep 18 06:40:25.118004 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:25.117974    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7kr9\" (UniqueName: \"kubernetes.io/projected/264e4a01-6af3-4613-8d1b-f55b0aa1af3b-kube-api-access-h7kr9\") pod \"test-curl-7765-d25e23\" (UID: \"264e4a01-6af3-4613-8d1b-f55b0aa1af3b\") " pod="opendatahub/test-curl-7765-d25e23"
Sep 18 06:40:25.121552 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:25.121526    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b" event={"ID":"d88eebd0-4865-4103-ab85-92c6c00cbd74","Type":"ContainerStarted","Data":"214a428712c0451bd823faa0cb3a7739e7cd83185d5cc65d3d03bedbead4be01"}
Sep 18 06:40:25.142475 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:25.142418    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b" podStartSLOduration=2.142397577 podStartE2EDuration="2.142397577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:40:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:40:25.138522172 +0000 UTC m=+1090.820049915" watchObservedRunningTime="2026-09-18 06:40:25.142397577 +0000 UTC m=+1090.823925323"
Sep 18 06:40:25.218922 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:25.218835    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7kr9\" (UniqueName: \"kubernetes.io/projected/264e4a01-6af3-4613-8d1b-f55b0aa1af3b-kube-api-access-h7kr9\") pod \"test-curl-7765-d25e23\" (UID: \"264e4a01-6af3-4613-8d1b-f55b0aa1af3b\") " pod="opendatahub/test-curl-7765-d25e23"
Sep 18 06:40:25.226234 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:25.226190    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7kr9\" (UniqueName: \"kubernetes.io/projected/264e4a01-6af3-4613-8d1b-f55b0aa1af3b-kube-api-access-h7kr9\") pod \"test-curl-7765-d25e23\" (UID: \"264e4a01-6af3-4613-8d1b-f55b0aa1af3b\") " pod="opendatahub/test-curl-7765-d25e23"
Sep 18 06:40:25.283979 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:25.283951    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7765-d25e23"
Sep 18 06:40:25.554484 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:25.554462    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7765-d25e23"]
Sep 18 06:40:25.555789 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:40:25.555761    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod264e4a01_6af3_4613_8d1b_f55b0aa1af3b.slice/crio-7e24201a5ad8439b3d522e7840c7f1dfbbafabc2eb9a83fa0f74236e85564540 WatchSource:0}: Error finding container 7e24201a5ad8439b3d522e7840c7f1dfbbafabc2eb9a83fa0f74236e85564540: Status 404 returned error can't find the container with id 7e24201a5ad8439b3d522e7840c7f1dfbbafabc2eb9a83fa0f74236e85564540
Sep 18 06:40:25.780287 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:25.780255    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:25.784983 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:25.784957    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:26.127820 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:26.127781    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7765-d25e23" event={"ID":"264e4a01-6af3-4613-8d1b-f55b0aa1af3b","Type":"ContainerStarted","Data":"7e24201a5ad8439b3d522e7840c7f1dfbbafabc2eb9a83fa0f74236e85564540"}
Sep 18 06:40:26.128271 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:26.128111    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:26.129124 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:26.129100    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:26.331189 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:26.331146    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:26.406828 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:26.406791    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:26.524143 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:26.524108    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:27.004406 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:27.004368    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:27.099171 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:27.099139    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:40:27.100767 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:27.100733    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:27.975204 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:27.975164    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:28.036938 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:28.036908    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:28.137370 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:28.137336    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:28.141475 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:28.141443    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7765-d25e23" event={"ID":"264e4a01-6af3-4613-8d1b-f55b0aa1af3b","Type":"ContainerStarted","Data":"3f65d9ecc2545ff820d3ecc8fb9dd9d72e25e99978f8c608b712c144489b45a5"}
Sep 18 06:40:28.154938 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:28.154896    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7765-d25e23" podStartSLOduration=2.289194161 podStartE2EDuration="4.154882357s" totalImagesPullingTime="1.865688196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:40:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:40:28.154170633 +0000 UTC m=+1093.835698381" watchObservedRunningTime="2026-09-18 06:40:28.154882357 +0000 UTC m=+1093.836410100"
Sep 18 06:40:28.555848 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:28.555717    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:28.662045 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:28.661987    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:28.721421 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:28.721390    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7765-d25e23"]
Sep 18 06:40:28.725128 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:28.725104    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7765-d25e23"]
Sep 18 06:40:28.835809 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:28.835718    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7765-3697dc"]
Sep 18 06:40:28.836206 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:28.836188    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="264e4a01-6af3-4613-8d1b-f55b0aa1af3b" containerName="test-curl-7765-d25e23"
Sep 18 06:40:28.836335 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:28.836209    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="264e4a01-6af3-4613-8d1b-f55b0aa1af3b" containerName="test-curl-7765-d25e23"
Sep 18 06:40:28.836335 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:28.836330    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="264e4a01-6af3-4613-8d1b-f55b0aa1af3b" containerName="test-curl-7765-d25e23"
Sep 18 06:40:28.839614 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:28.839587    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7765-3697dc"
Sep 18 06:40:28.845156 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:28.845109    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7765-3697dc"]
Sep 18 06:40:28.944190 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:28.944156    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:28.955459 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:28.955434    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj69r\" (UniqueName: \"kubernetes.io/projected/fbd5a348-80ae-46ef-adeb-31408b0582e4-kube-api-access-xj69r\") pod \"test-curl-7765-3697dc\" (UID: \"fbd5a348-80ae-46ef-adeb-31408b0582e4\") " pod="opendatahub/test-curl-7765-3697dc"
Sep 18 06:40:28.993519 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:28.993470    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:29.056751 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:29.056716    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj69r\" (UniqueName: \"kubernetes.io/projected/fbd5a348-80ae-46ef-adeb-31408b0582e4-kube-api-access-xj69r\") pod \"test-curl-7765-3697dc\" (UID: \"fbd5a348-80ae-46ef-adeb-31408b0582e4\") " pod="opendatahub/test-curl-7765-3697dc"
Sep 18 06:40:29.064107 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:29.064075    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj69r\" (UniqueName: \"kubernetes.io/projected/fbd5a348-80ae-46ef-adeb-31408b0582e4-kube-api-access-xj69r\") pod \"test-curl-7765-3697dc\" (UID: \"fbd5a348-80ae-46ef-adeb-31408b0582e4\") " pod="opendatahub/test-curl-7765-3697dc"
Sep 18 06:40:29.146353 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:40:29.146279    2568 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7765-d25e23" container="test-curl-7765-d25e23"
Sep 18 06:40:29.157304 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:29.157280    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7765-3697dc"
Sep 18 06:40:29.330577 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:29.330531    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:29.428038 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:29.428015    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7765-3697dc"]
Sep 18 06:40:29.430961 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:29.430931    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:29.434116 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:40:29.434077    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfbd5a348_80ae_46ef_adeb_31408b0582e4.slice/crio-104d67f8f9102dab231b1fb9647def3b63865d8fc5b53722429f1f35477edffb WatchSource:0}: Error finding container 104d67f8f9102dab231b1fb9647def3b63865d8fc5b53722429f1f35477edffb: Status 404 returned error can't find the container with id 104d67f8f9102dab231b1fb9647def3b63865d8fc5b53722429f1f35477edffb
Sep 18 06:40:29.495326 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:29.495297    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:29.900043 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:29.900009    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"]
Sep 18 06:40:29.904084 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:29.904060    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"
Sep 18 06:40:29.906530 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:29.906509    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-946609-serving-cert\""
Sep 18 06:40:29.906738 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:29.906629    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 18 06:40:29.907809 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:29.907618    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-mxmzc\""
Sep 18 06:40:29.907809 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:29.907683    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 18 06:40:29.907809 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:29.907691    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 06:40:29.915799 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:29.915574    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"]
Sep 18 06:40:29.964021 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:29.963990    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea698d13-b785-4a73-8563-04df09ce4424-metrics-certs\") pod \"maas-api-e2e-worker-w0-946609-9986b49d-c5r4f\" (UID: \"ea698d13-b785-4a73-8563-04df09ce4424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"
Sep 18 06:40:29.964208 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:29.964125    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea698d13-b785-4a73-8563-04df09ce4424-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-946609-9986b49d-c5r4f\" (UID: \"ea698d13-b785-4a73-8563-04df09ce4424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"
Sep 18 06:40:29.964208 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:29.964159    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea698d13-b785-4a73-8563-04df09ce4424-maas-api-tls\") pod \"maas-api-e2e-worker-w0-946609-9986b49d-c5r4f\" (UID: \"ea698d13-b785-4a73-8563-04df09ce4424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"
Sep 18 06:40:29.964208 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:29.964182    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jdjh\" (UniqueName: \"kubernetes.io/projected/ea698d13-b785-4a73-8563-04df09ce4424-kube-api-access-7jdjh\") pod \"maas-api-e2e-worker-w0-946609-9986b49d-c5r4f\" (UID: \"ea698d13-b785-4a73-8563-04df09ce4424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"
Sep 18 06:40:29.975680 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:29.975643    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:30.064912 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:30.064879    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea698d13-b785-4a73-8563-04df09ce4424-metrics-certs\") pod \"maas-api-e2e-worker-w0-946609-9986b49d-c5r4f\" (UID: \"ea698d13-b785-4a73-8563-04df09ce4424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"
Sep 18 06:40:30.065402 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:30.064960    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea698d13-b785-4a73-8563-04df09ce4424-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-946609-9986b49d-c5r4f\" (UID: \"ea698d13-b785-4a73-8563-04df09ce4424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"
Sep 18 06:40:30.065402 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:30.064982    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea698d13-b785-4a73-8563-04df09ce4424-maas-api-tls\") pod \"maas-api-e2e-worker-w0-946609-9986b49d-c5r4f\" (UID: \"ea698d13-b785-4a73-8563-04df09ce4424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"
Sep 18 06:40:30.065402 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:30.064999    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jdjh\" (UniqueName: \"kubernetes.io/projected/ea698d13-b785-4a73-8563-04df09ce4424-kube-api-access-7jdjh\") pod \"maas-api-e2e-worker-w0-946609-9986b49d-c5r4f\" (UID: \"ea698d13-b785-4a73-8563-04df09ce4424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"
Sep 18 06:40:30.065402 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:40:30.065117    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-serving-cert: secret "maas-api-e2e-worker-w0-946609-serving-cert" not found
Sep 18 06:40:30.065402 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:40:30.065194    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea698d13-b785-4a73-8563-04df09ce4424-maas-api-tls podName:ea698d13-b785-4a73-8563-04df09ce4424 nodeName:}" failed. No retries permitted until 2026-09-18 06:40:30.565174945 +0000 UTC m=+1096.246702668 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ea698d13-b785-4a73-8563-04df09ce4424-maas-api-tls") pod "maas-api-e2e-worker-w0-946609-9986b49d-c5r4f" (UID: "ea698d13-b785-4a73-8563-04df09ce4424") : secret "maas-api-e2e-worker-w0-946609-serving-cert" not found
Sep 18 06:40:30.065716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:30.065697    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea698d13-b785-4a73-8563-04df09ce4424-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-946609-9986b49d-c5r4f\" (UID: \"ea698d13-b785-4a73-8563-04df09ce4424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"
Sep 18 06:40:30.067417 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:30.067398    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea698d13-b785-4a73-8563-04df09ce4424-metrics-certs\") pod \"maas-api-e2e-worker-w0-946609-9986b49d-c5r4f\" (UID: \"ea698d13-b785-4a73-8563-04df09ce4424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"
Sep 18 06:40:30.075441 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:30.075421    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jdjh\" (UniqueName: \"kubernetes.io/projected/ea698d13-b785-4a73-8563-04df09ce4424-kube-api-access-7jdjh\") pod \"maas-api-e2e-worker-w0-946609-9986b49d-c5r4f\" (UID: \"ea698d13-b785-4a73-8563-04df09ce4424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"
Sep 18 06:40:30.076068 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:30.076044    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:30.153009 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:30.152921    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7765-3697dc" event={"ID":"fbd5a348-80ae-46ef-adeb-31408b0582e4","Type":"ContainerStarted","Data":"63d18706b86073fb0800829d9d187747ff01b9b1d30bc1795552ea18ce8a136c"}
Sep 18 06:40:30.153009 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:30.152969    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7765-3697dc" event={"ID":"fbd5a348-80ae-46ef-adeb-31408b0582e4","Type":"ContainerStarted","Data":"104d67f8f9102dab231b1fb9647def3b63865d8fc5b53722429f1f35477edffb"}
Sep 18 06:40:30.153267 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:30.153037    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7765-d25e23" podUID="264e4a01-6af3-4613-8d1b-f55b0aa1af3b" containerName="test-curl-7765-d25e23" containerID="cri-o://3f65d9ecc2545ff820d3ecc8fb9dd9d72e25e99978f8c608b712c144489b45a5" gracePeriod=2
Sep 18 06:40:30.155338 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:40:30.155308    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7765-d25e23\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="264e4a01-6af3-4613-8d1b-f55b0aa1af3b" pod="opendatahub/test-curl-7765-d25e23"
Sep 18 06:40:30.168664 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:30.168620    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7765-3697dc" podStartSLOduration=2.168606673 podStartE2EDuration="2.168606673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:40:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:40:30.167664663 +0000 UTC m=+1095.849192410" watchObservedRunningTime="2026-09-18 06:40:30.168606673 +0000 UTC m=+1095.850134417"
Sep 18 06:40:30.169650 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:40:30.169625    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7765-d25e23\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="264e4a01-6af3-4613-8d1b-f55b0aa1af3b" pod="opendatahub/test-curl-7765-d25e23"
Sep 18 06:40:30.269381 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:30.269354    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:30.486639 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:30.486542    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:30.569972 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:30.569943    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea698d13-b785-4a73-8563-04df09ce4424-maas-api-tls\") pod \"maas-api-e2e-worker-w0-946609-9986b49d-c5r4f\" (UID: \"ea698d13-b785-4a73-8563-04df09ce4424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"
Sep 18 06:40:30.572315 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:30.572298    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea698d13-b785-4a73-8563-04df09ce4424-maas-api-tls\") pod \"maas-api-e2e-worker-w0-946609-9986b49d-c5r4f\" (UID: \"ea698d13-b785-4a73-8563-04df09ce4424\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"
Sep 18 06:40:30.586932 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:30.586900    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:30.790193 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:30.790116    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7765-3697dc"]
Sep 18 06:40:30.795064 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:30.795039    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7765-3697dc"]
Sep 18 06:40:30.818230 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:30.818192    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"
Sep 18 06:40:30.870556 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:30.869888    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:31.092198 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.092174    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"]
Sep 18 06:40:31.093985 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:40:31.093953    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea698d13_b785_4a73_8563_04df09ce4424.slice/crio-77eafbb1c78d4f7d9f5264ccd759e1ef01eb825f20f575fe5da8ba15db1285a6 WatchSource:0}: Error finding container 77eafbb1c78d4f7d9f5264ccd759e1ef01eb825f20f575fe5da8ba15db1285a6: Status 404 returned error can't find the container with id 77eafbb1c78d4f7d9f5264ccd759e1ef01eb825f20f575fe5da8ba15db1285a6
Sep 18 06:40:31.158908 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.158872    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f" event={"ID":"ea698d13-b785-4a73-8563-04df09ce4424","Type":"ContainerStarted","Data":"77eafbb1c78d4f7d9f5264ccd759e1ef01eb825f20f575fe5da8ba15db1285a6"}
Sep 18 06:40:31.159209 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:40:31.159191    2568 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7765-3697dc" container="test-curl-7765-3697dc"
Sep 18 06:40:31.280529 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.280498    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:31.603071 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.603038    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"]
Sep 18 06:40:31.603672 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.603649    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbd5a348-80ae-46ef-adeb-31408b0582e4" containerName="test-curl-7765-3697dc"
Sep 18 06:40:31.603672 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.603673    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbd5a348-80ae-46ef-adeb-31408b0582e4" containerName="test-curl-7765-3697dc"
Sep 18 06:40:31.603876 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.603769    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbd5a348-80ae-46ef-adeb-31408b0582e4" containerName="test-curl-7765-3697dc"
Sep 18 06:40:31.609791 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.607044    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.611166 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.611141    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-43fdfe-openshift-default-dockercfg-rzdf6\""
Sep 18 06:40:31.616507 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.616484    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"]
Sep 18 06:40:31.628100 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.627966    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:31.680686 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.680653    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-workload-socket\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.680872 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.680726    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-envoy\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.680872 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.680776    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-data\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.680872 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.680795    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-podinfo\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.680872 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.680827    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f6c5262-48e2-42b1-9e20-45fe353bc292-istiod-ca-cert\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.680872 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.680855    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-token\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.681156 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.680960    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-credential-socket\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.681156 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.681055    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-workload-certs\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.681156 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.681095    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78c22\" (UniqueName: \"kubernetes.io/projected/0f6c5262-48e2-42b1-9e20-45fe353bc292-kube-api-access-78c22\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.724543 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.724503    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:31.782629 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.782579    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-envoy\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.782799 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.782684    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-data\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.782799 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.782726    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-podinfo\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.782799 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.782756    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f6c5262-48e2-42b1-9e20-45fe353bc292-istiod-ca-cert\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.782799 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.782782    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-token\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.783016 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.782861    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-credential-socket\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.783016 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.782917    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-workload-certs\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.783016 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.782945    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78c22\" (UniqueName: \"kubernetes.io/projected/0f6c5262-48e2-42b1-9e20-45fe353bc292-kube-api-access-78c22\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.783016 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.783004    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-workload-socket\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.783371 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.783347    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-data\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.783468 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.783448    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-credential-socket\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.783583 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.783561    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-workload-certs\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.783687 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.783611    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-workload-socket\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.783687 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.783667    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f6c5262-48e2-42b1-9e20-45fe353bc292-istiod-ca-cert\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.785373 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.785330    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-envoy\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.785373 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.785368    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-podinfo\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.790786 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.790740    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78c22\" (UniqueName: \"kubernetes.io/projected/0f6c5262-48e2-42b1-9e20-45fe353bc292-kube-api-access-78c22\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.790893 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.790814    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-token\") pod \"e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") " pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.824818 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.824789    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:31.923179 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.923104    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:31.924713 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:31.924664    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:32.025188 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.025162    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:32.142625 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.141499    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"]
Sep 18 06:40:32.150236 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.148394    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.151451 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.151111    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0da33e9c-openshift-default-dockercfg-g9pth\""
Sep 18 06:40:32.160069 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.160011    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"]
Sep 18 06:40:32.171330 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.166964    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:32.183519 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.183483    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7765-3697dc" podUID="fbd5a348-80ae-46ef-adeb-31408b0582e4" containerName="test-curl-7765-3697dc" containerID="cri-o://63d18706b86073fb0800829d9d187747ff01b9b1d30bc1795552ea18ce8a136c" gracePeriod=2
Sep 18 06:40:32.186410 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:40:32.186273    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7765-3697dc\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="fbd5a348-80ae-46ef-adeb-31408b0582e4" pod="opendatahub/test-curl-7765-3697dc"
Sep 18 06:40:32.226846 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.226813    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"]
Sep 18 06:40:32.230611 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:40:32.229464    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f6c5262_48e2_42b1_9e20_45fe353bc292.slice/crio-d004cfde3a257241456b187fbb8b5870f0e360074bd12f54414aada33c82c51e WatchSource:0}: Error finding container d004cfde3a257241456b187fbb8b5870f0e360074bd12f54414aada33c82c51e: Status 404 returned error can't find the container with id d004cfde3a257241456b187fbb8b5870f0e360074bd12f54414aada33c82c51e
Sep 18 06:40:32.240952 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.240377    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:40:32.240952 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.240477    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:40:32.240952 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.240517    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:40:32.288525 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.288491    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istiod-ca-cert\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.288689 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.288560    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-token\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.288689 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.288590    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-envoy\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.288689 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.288623    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-credential-socket\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.288864 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.288788    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-workload-certs\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.288864 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.288830    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-workload-socket\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.288968 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.288863    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-data\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.288968 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.288928    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-podinfo\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.288968 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.288954    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m44g4\" (UniqueName: \"kubernetes.io/projected/3764aa9c-4226-414b-a644-01a8e3b4b0a8-kube-api-access-m44g4\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.364886 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.364455    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:32.390579 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.389867    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-workload-socket\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.390579 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.389903    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-data\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.390579 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.389955    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-podinfo\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.390579 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.389986    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m44g4\" (UniqueName: \"kubernetes.io/projected/3764aa9c-4226-414b-a644-01a8e3b4b0a8-kube-api-access-m44g4\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.390579 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.390050    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istiod-ca-cert\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.390579 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.390092    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-token\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.390579 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.390119    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-envoy\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.390579 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.390148    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-credential-socket\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.390579 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.390189    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-workload-certs\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.390579 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.390292    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-workload-socket\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.390579 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.390338    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-data\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.390579 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.390536    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-workload-certs\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.391325 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.390953    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istiod-ca-cert\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.391325 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.391251    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-credential-socket\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.392972 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.392929    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-envoy\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.393859 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.393835    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-podinfo\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.398743 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.398704    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-token\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.399195 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.399174    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m44g4\" (UniqueName: \"kubernetes.io/projected/3764aa9c-4226-414b-a644-01a8e3b4b0a8-kube-api-access-m44g4\") pod \"e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") " pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.412824 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.412799    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7765-d25e23"
Sep 18 06:40:32.414979 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:40:32.414946    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7765-3697dc\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="fbd5a348-80ae-46ef-adeb-31408b0582e4" pod="opendatahub/test-curl-7765-3697dc"
Sep 18 06:40:32.463149 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.463068    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:32.491414 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.491377    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7kr9\" (UniqueName: \"kubernetes.io/projected/264e4a01-6af3-4613-8d1b-f55b0aa1af3b-kube-api-access-h7kr9\") pod \"264e4a01-6af3-4613-8d1b-f55b0aa1af3b\" (UID: \"264e4a01-6af3-4613-8d1b-f55b0aa1af3b\") "
Sep 18 06:40:32.494157 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.494110    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/264e4a01-6af3-4613-8d1b-f55b0aa1af3b-kube-api-access-h7kr9" (OuterVolumeSpecName: "kube-api-access-h7kr9") pod "264e4a01-6af3-4613-8d1b-f55b0aa1af3b" (UID: "264e4a01-6af3-4613-8d1b-f55b0aa1af3b"). InnerVolumeSpecName "kube-api-access-h7kr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:40:32.524435 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.524399    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:32.547485 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.547066    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="264e4a01-6af3-4613-8d1b-f55b0aa1af3b" path="/var/lib/kubelet/pods/264e4a01-6af3-4613-8d1b-f55b0aa1af3b/volumes"
Sep 18 06:40:32.594147 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.594101    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7kr9\" (UniqueName: \"kubernetes.io/projected/264e4a01-6af3-4613-8d1b-f55b0aa1af3b-kube-api-access-h7kr9\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:40:32.814533 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:32.814501    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"]
Sep 18 06:40:33.124385 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:33.124306    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:33.194849 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:33.194775    2568 generic.go:356] "Generic (PLEG): container finished" podID="264e4a01-6af3-4613-8d1b-f55b0aa1af3b" containerID="3f65d9ecc2545ff820d3ecc8fb9dd9d72e25e99978f8c608b712c144489b45a5" exitCode=137
Sep 18 06:40:33.194849 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:33.194834    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7765-d25e23"
Sep 18 06:40:33.194849 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:33.194848    2568 scope.go:117] "RemoveContainer" containerID="3f65d9ecc2545ff820d3ecc8fb9dd9d72e25e99978f8c608b712c144489b45a5"
Sep 18 06:40:33.198162 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:33.198046    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts" event={"ID":"0f6c5262-48e2-42b1-9e20-45fe353bc292","Type":"ContainerStarted","Data":"3e8a27f6d405745b8130f7951277864691983c05e151a19a4c0eb8f439eb7f5c"}
Sep 18 06:40:33.198162 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:33.198108    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts" event={"ID":"0f6c5262-48e2-42b1-9e20-45fe353bc292","Type":"ContainerStarted","Data":"d004cfde3a257241456b187fbb8b5870f0e360074bd12f54414aada33c82c51e"}
Sep 18 06:40:33.218058 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:33.218003    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts" podStartSLOduration=2.217984747 podStartE2EDuration="2.217984747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:40:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:40:33.216793798 +0000 UTC m=+1098.898321545" watchObservedRunningTime="2026-09-18 06:40:33.217984747 +0000 UTC m=+1098.899512485"
Sep 18 06:40:33.786778 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:33.786689    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:33.802992 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:40:33.802949    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3764aa9c_4226_414b_a644_01a8e3b4b0a8.slice/crio-a7a3b227094e3194c0b9a2fe0816d98c4e73c7c3cdb81de0b87fbbeafb69dc44 WatchSource:0}: Error finding container a7a3b227094e3194c0b9a2fe0816d98c4e73c7c3cdb81de0b87fbbeafb69dc44: Status 404 returned error can't find the container with id a7a3b227094e3194c0b9a2fe0816d98c4e73c7c3cdb81de0b87fbbeafb69dc44
Sep 18 06:40:33.805487 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:33.805462    2568 scope.go:117] "RemoveContainer" containerID="3f65d9ecc2545ff820d3ecc8fb9dd9d72e25e99978f8c608b712c144489b45a5"
Sep 18 06:40:33.805710 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:33.805679    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:40:33.805787 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:33.805764    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:40:33.805787 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:40:33.805776    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f65d9ecc2545ff820d3ecc8fb9dd9d72e25e99978f8c608b712c144489b45a5\": container with ID starting with 3f65d9ecc2545ff820d3ecc8fb9dd9d72e25e99978f8c608b712c144489b45a5 not found: ID does not exist" containerID="3f65d9ecc2545ff820d3ecc8fb9dd9d72e25e99978f8c608b712c144489b45a5"
Sep 18 06:40:33.805902 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:33.805804    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f65d9ecc2545ff820d3ecc8fb9dd9d72e25e99978f8c608b712c144489b45a5"} err="failed to get container status \"3f65d9ecc2545ff820d3ecc8fb9dd9d72e25e99978f8c608b712c144489b45a5\": rpc error: code = NotFound desc = could not find container \"3f65d9ecc2545ff820d3ecc8fb9dd9d72e25e99978f8c608b712c144489b45a5\": container with ID starting with 3f65d9ecc2545ff820d3ecc8fb9dd9d72e25e99978f8c608b712c144489b45a5 not found: ID does not exist"
Sep 18 06:40:33.805902 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:33.805810    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:40:33.924007 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:33.923979    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:33.933247 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:33.929286    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:34.205059 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:34.205027    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd" event={"ID":"3764aa9c-4226-414b-a644-01a8e3b4b0a8","Type":"ContainerStarted","Data":"56a31fcbcf34e4e9bd74c8cd49b511347fdc6b8726faee529579ea11a3b5c408"}
Sep 18 06:40:34.205443 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:34.205084    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd" event={"ID":"3764aa9c-4226-414b-a644-01a8e3b4b0a8","Type":"ContainerStarted","Data":"a7a3b227094e3194c0b9a2fe0816d98c4e73c7c3cdb81de0b87fbbeafb69dc44"}
Sep 18 06:40:34.206532 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:34.206507    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f" event={"ID":"ea698d13-b785-4a73-8563-04df09ce4424","Type":"ContainerStarted","Data":"60811aded78ad3264eebbee762843413185aaefbda244276ba68e80d0e013bb8"}
Sep 18 06:40:34.206741 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:34.206724    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:34.207638 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:34.207611    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:40:34.223935 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:34.223888    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd" podStartSLOduration=2.223874209 podStartE2EDuration="2.223874209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:40:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:40:34.221808001 +0000 UTC m=+1099.903335828" watchObservedRunningTime="2026-09-18 06:40:34.223874209 +0000 UTC m=+1099.905401952"
Sep 18 06:40:34.261960 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:34.261893    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f" podStartSLOduration=2.489252916 podStartE2EDuration="5.261869285s" totalImagesPullingTime="2.772616369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:40:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:40:34.25847976 +0000 UTC m=+1099.940007506" watchObservedRunningTime="2026-09-18 06:40:34.261869285 +0000 UTC m=+1099.943397037"
Sep 18 06:40:34.367939 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:34.367917    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7765-3697dc"
Sep 18 06:40:34.515894 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:34.515807    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xj69r\" (UniqueName: \"kubernetes.io/projected/fbd5a348-80ae-46ef-adeb-31408b0582e4-kube-api-access-xj69r\") pod \"fbd5a348-80ae-46ef-adeb-31408b0582e4\" (UID: \"fbd5a348-80ae-46ef-adeb-31408b0582e4\") "
Sep 18 06:40:34.517820 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:34.517789    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbd5a348-80ae-46ef-adeb-31408b0582e4-kube-api-access-xj69r" (OuterVolumeSpecName: "kube-api-access-xj69r") pod "fbd5a348-80ae-46ef-adeb-31408b0582e4" (UID: "fbd5a348-80ae-46ef-adeb-31408b0582e4"). InnerVolumeSpecName "kube-api-access-xj69r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:40:34.524787 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:34.524766    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:34.530471 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:34.530448    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:34.543103 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:34.543080    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbd5a348-80ae-46ef-adeb-31408b0582e4" path="/var/lib/kubelet/pods/fbd5a348-80ae-46ef-adeb-31408b0582e4/volumes"
Sep 18 06:40:34.616854 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:34.616818    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xj69r\" (UniqueName: \"kubernetes.io/projected/fbd5a348-80ae-46ef-adeb-31408b0582e4-kube-api-access-xj69r\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:40:34.732138 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:34.732088    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:35.064489 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:35.064447    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:35.160905 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:35.160874    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:35.212342 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:35.212309    2568 generic.go:356] "Generic (PLEG): container finished" podID="fbd5a348-80ae-46ef-adeb-31408b0582e4" containerID="63d18706b86073fb0800829d9d187747ff01b9b1d30bc1795552ea18ce8a136c" exitCode=137
Sep 18 06:40:35.212747 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:35.212355    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7765-3697dc"
Sep 18 06:40:35.212747 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:35.212417    2568 scope.go:117] "RemoveContainer" containerID="63d18706b86073fb0800829d9d187747ff01b9b1d30bc1795552ea18ce8a136c"
Sep 18 06:40:35.212747 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:35.212608    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"
Sep 18 06:40:35.213143 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:35.213109    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:35.214653 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:35.214457    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:40:35.228381 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:35.228358    2568 scope.go:117] "RemoveContainer" containerID="63d18706b86073fb0800829d9d187747ff01b9b1d30bc1795552ea18ce8a136c"
Sep 18 06:40:35.228849 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:40:35.228829    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63d18706b86073fb0800829d9d187747ff01b9b1d30bc1795552ea18ce8a136c\": container with ID starting with 63d18706b86073fb0800829d9d187747ff01b9b1d30bc1795552ea18ce8a136c not found: ID does not exist" containerID="63d18706b86073fb0800829d9d187747ff01b9b1d30bc1795552ea18ce8a136c"
Sep 18 06:40:35.228934 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:35.228855    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63d18706b86073fb0800829d9d187747ff01b9b1d30bc1795552ea18ce8a136c"} err="failed to get container status \"63d18706b86073fb0800829d9d187747ff01b9b1d30bc1795552ea18ce8a136c\": rpc error: code = NotFound desc = could not find container \"63d18706b86073fb0800829d9d187747ff01b9b1d30bc1795552ea18ce8a136c\": container with ID starting with 63d18706b86073fb0800829d9d187747ff01b9b1d30bc1795552ea18ce8a136c not found: ID does not exist"
Sep 18 06:40:35.737740 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:35.737710    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:35.841754 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:35.841722    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:35.937267 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:35.937234    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:36.863783 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:36.863747    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:37.708636 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:37.708608    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:38.433617 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:38.433584    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:38.544698 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:38.544667    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:38.666778 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:38.666728    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:38.873127 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:38.873024    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:38.973715 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:38.973682    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:39.246620 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:39.246586    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:40.042242 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:40.041601    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:40.141869 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:40.141833    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:40.534660 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:40.534619    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:40.629625 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:40.629593    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:40.716784 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:40.716755    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:40.945287 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:40.945237    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:41.046942 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:41.046911    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:41.224088 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:41.224011    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"
Sep 18 06:40:41.442309 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:41.442242    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:41.515817 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:41.515733    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:41.622330 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:41.622287    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:41.821026 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:41.820940    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:42.085599 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:42.085513    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:42.533483 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:42.533446    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:43.308355 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:43.308321    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:43.578038 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:43.577949    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:43.731929 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:43.731892    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:43.850471 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:43.850393    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:43.937447 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:43.937406    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:44.244296 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:44.244263    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:44.853821 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:44.853791    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:45.629032 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:45.628998    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:46.132206 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.132121    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:46.284011 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.283927    2568 generic.go:356] "Generic (PLEG): container finished" podID="d88eebd0-4865-4103-ab85-92c6c00cbd74" containerID="214a428712c0451bd823faa0cb3a7739e7cd83185d5cc65d3d03bedbead4be01" exitCode=0
Sep 18 06:40:46.284011 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.284002    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b" event={"ID":"d88eebd0-4865-4103-ab85-92c6c00cbd74","Type":"ContainerDied","Data":"214a428712c0451bd823faa0cb3a7739e7cd83185d5cc65d3d03bedbead4be01"}
Sep 18 06:40:46.284470 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.284456    2568 scope.go:117] "RemoveContainer" containerID="214a428712c0451bd823faa0cb3a7739e7cd83185d5cc65d3d03bedbead4be01"
Sep 18 06:40:46.285636 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.285609    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:40:46.285689 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.285679    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:40:46.285724 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.285709    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:40:46.344714 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.344688    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:46.450210 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.450183    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:46.578344 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.578268    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:46.717145 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.717114    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"]
Sep 18 06:40:46.721671 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.721648    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.724447 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.724423    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-708b6f-openshift-default-dockercfg-k5w2c\""
Sep 18 06:40:46.735833 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.735730    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"]
Sep 18 06:40:46.747116 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.747086    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-data\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.747334 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.747311    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn49c\" (UniqueName: \"kubernetes.io/projected/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-kube-api-access-wn49c\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.747486 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.747468    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-token\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.747670 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.747635    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-envoy\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.747798 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.747782    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-credential-socket\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.747947 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.747931    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istiod-ca-cert\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.748076 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.748061    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-workload-certs\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.748207 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.748191    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-workload-socket\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.748382 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.748366    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-podinfo\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.750971 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.750952    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:46.849929 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.849432    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn49c\" (UniqueName: \"kubernetes.io/projected/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-kube-api-access-wn49c\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.849929 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.849900    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-token\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.850340 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.850319    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-envoy\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.850792 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.850361    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-credential-socket\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.850853 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.850833    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-credential-socket\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.850955 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.850935    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istiod-ca-cert\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.851013 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.850980    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-workload-certs\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.851070 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.851037    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-workload-socket\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.851070 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.851062    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-podinfo\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.851191 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.851176    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-data\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.851780 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.851755    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istiod-ca-cert\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.852178 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.851998    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-data\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.852178 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.852110    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-workload-certs\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.852591 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.852564    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-workload-socket\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.854796 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.854754    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-envoy\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.857926 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.857754    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-token\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.858749 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.858726    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-podinfo\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:46.859495 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:46.859448    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn49c\" (UniqueName: \"kubernetes.io/projected/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-kube-api-access-wn49c\") pod \"e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") " pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:47.039930 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:47.039894    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:47.132321 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:47.132234    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:47.232795 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:47.232756    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:47.291552 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:47.291520    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b" event={"ID":"d88eebd0-4865-4103-ab85-92c6c00cbd74","Type":"ContainerStarted","Data":"014ee7475637d59e50fde608c06c418d3f4afef86c32fbf534648bc254472351"}
Sep 18 06:40:47.320385 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:47.320359    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"]
Sep 18 06:40:47.322325 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:40:47.322299    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96c8cb03_28cf_41cb_829d_a7b9073bdb1c.slice/crio-ef54da09fc4c67dda0a827831f2e136374359741de1db6995aba2f155b085ce7 WatchSource:0}: Error finding container ef54da09fc4c67dda0a827831f2e136374359741de1db6995aba2f155b085ce7: Status 404 returned error can't find the container with id ef54da09fc4c67dda0a827831f2e136374359741de1db6995aba2f155b085ce7
Sep 18 06:40:47.324660 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:47.324630    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:40:47.324776 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:47.324691    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:40:47.324776 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:47.324718    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:40:47.779935 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:47.779901    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:47.780256 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:47.780237    2568 patch_prober.go:28] interesting pod/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 18 06:40:47.780314 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:47.780281    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b" podUID="d88eebd0-4865-4103-ab85-92c6c00cbd74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 18 06:40:47.930035 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:47.929993    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:48.085103 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:48.085023    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:48.298321 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:48.298283    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b" event={"ID":"96c8cb03-28cf-41cb-829d-a7b9073bdb1c","Type":"ContainerStarted","Data":"b3266bfbf38d05db250319aa0dd14a0a8ce457454a807ad0a57a8b316afca9e6"}
Sep 18 06:40:48.298698 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:48.298342    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b" event={"ID":"96c8cb03-28cf-41cb-829d-a7b9073bdb1c","Type":"ContainerStarted","Data":"ef54da09fc4c67dda0a827831f2e136374359741de1db6995aba2f155b085ce7"}
Sep 18 06:40:48.317770 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:48.317723    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b" podStartSLOduration=2.3177072499999998 podStartE2EDuration="2.31770725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:40:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:40:48.315728935 +0000 UTC m=+1113.997256679" watchObservedRunningTime="2026-09-18 06:40:48.31770725 +0000 UTC m=+1113.999234994"
Sep 18 06:40:48.592364 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:48.592329    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:48.746596 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:48.746512    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:48.780674 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:48.780638    2568 patch_prober.go:28] interesting pod/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 18 06:40:48.780831 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:48.780694    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b" podUID="d88eebd0-4865-4103-ab85-92c6c00cbd74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 18 06:40:48.827978 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:48.827948    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:40:49.007298 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.007260    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"]
Sep 18 06:40:49.013198 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.013175    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:49.015692 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.015668    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-654463\"/\"openshift-service-ca.crt\""
Sep 18 06:40:49.015831 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.015717    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-654463\"/\"default-dockercfg-f7j4n\""
Sep 18 06:40:49.015831 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.015744    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-654463\"/\"kube-root-ca.crt\""
Sep 18 06:40:49.016090 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.015891    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-654463\"/\"simulator-w4-654463-kserve-self-signed-certs\""
Sep 18 06:40:49.020568 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.020518    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"]
Sep 18 06:40:49.058243 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.053493    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:49.061353 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.060472    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:49.073418 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.073392    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrr9m\" (UniqueName: \"kubernetes.io/projected/871eb64e-4369-4bb5-915f-8c00f2adbbd1-kube-api-access-xrr9m\") pod \"simulator-w4-654463-kserve-87b6f7bf9-flt95\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") " pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:49.073543 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.073437    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-home\") pod \"simulator-w4-654463-kserve-87b6f7bf9-flt95\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") " pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:49.073543 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.073473    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/871eb64e-4369-4bb5-915f-8c00f2adbbd1-tls-certs\") pod \"simulator-w4-654463-kserve-87b6f7bf9-flt95\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") " pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:49.073666 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.073644    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-dshm\") pod \"simulator-w4-654463-kserve-87b6f7bf9-flt95\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") " pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:49.073771 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.073754    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-tmp-dir\") pod \"simulator-w4-654463-kserve-87b6f7bf9-flt95\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") " pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:49.073810 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.073798    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-model-cache\") pod \"simulator-w4-654463-kserve-87b6f7bf9-flt95\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") " pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:49.174417 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.174387    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-dshm\") pod \"simulator-w4-654463-kserve-87b6f7bf9-flt95\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") " pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:49.174585 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.174438    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-tmp-dir\") pod \"simulator-w4-654463-kserve-87b6f7bf9-flt95\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") " pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:49.174585 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.174464    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-model-cache\") pod \"simulator-w4-654463-kserve-87b6f7bf9-flt95\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") " pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:49.174585 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.174490    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrr9m\" (UniqueName: \"kubernetes.io/projected/871eb64e-4369-4bb5-915f-8c00f2adbbd1-kube-api-access-xrr9m\") pod \"simulator-w4-654463-kserve-87b6f7bf9-flt95\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") " pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:49.174585 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.174520    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-home\") pod \"simulator-w4-654463-kserve-87b6f7bf9-flt95\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") " pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:49.174585 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.174575    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/871eb64e-4369-4bb5-915f-8c00f2adbbd1-tls-certs\") pod \"simulator-w4-654463-kserve-87b6f7bf9-flt95\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") " pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:49.174954 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.174925    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-model-cache\") pod \"simulator-w4-654463-kserve-87b6f7bf9-flt95\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") " pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:49.175164 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.175140    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-home\") pod \"simulator-w4-654463-kserve-87b6f7bf9-flt95\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") " pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:49.175278 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.175206    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-tmp-dir\") pod \"simulator-w4-654463-kserve-87b6f7bf9-flt95\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") " pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:49.176978 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.176959    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/871eb64e-4369-4bb5-915f-8c00f2adbbd1-tls-certs\") pod \"simulator-w4-654463-kserve-87b6f7bf9-flt95\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") " pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:49.177143 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.177128    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-dshm\") pod \"simulator-w4-654463-kserve-87b6f7bf9-flt95\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") " pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:49.182234 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.182197    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrr9m\" (UniqueName: \"kubernetes.io/projected/871eb64e-4369-4bb5-915f-8c00f2adbbd1-kube-api-access-xrr9m\") pod \"simulator-w4-654463-kserve-87b6f7bf9-flt95\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") " pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:49.201853 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.201830    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:49.303502 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.303421    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:49.304294 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.304274    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:40:49.325076 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.325053    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:49.628998 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.628974    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"]
Sep 18 06:40:49.780385 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.780350    2568 patch_prober.go:28] interesting pod/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 18 06:40:49.780554 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.780418    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b" podUID="d88eebd0-4865-4103-ab85-92c6c00cbd74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 18 06:40:49.828495 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:49.828465    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:50.305938 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:50.305892    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:50.313016 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:50.312986    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95" event={"ID":"871eb64e-4369-4bb5-915f-8c00f2adbbd1","Type":"ContainerStarted","Data":"f0a9a5e63e0a674d4fd37f895d5e0ccc0d3602146ae8fc174227585c16d80d98"}
Sep 18 06:40:50.313185 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:50.313027    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95" event={"ID":"871eb64e-4369-4bb5-915f-8c00f2adbbd1","Type":"ContainerStarted","Data":"0b6be33758e38d00355e681e2304bcff993cff946bf88124a91b990768979e48"}
Sep 18 06:40:50.339794 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:50.339724    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95" podStartSLOduration=2.339701145 podStartE2EDuration="2.339701145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:40:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:40:50.339413651 +0000 UTC m=+1116.020941400" watchObservedRunningTime="2026-09-18 06:40:50.339701145 +0000 UTC m=+1116.021228890"
Sep 18 06:40:50.394446 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:50.394401    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:50.664429 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:50.664395    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:50.779879 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:50.779830    2568 patch_prober.go:28] interesting pod/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 18 06:40:50.780119 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:50.779913    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b" podUID="d88eebd0-4865-4103-ab85-92c6c00cbd74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 18 06:40:51.545655 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:51.545619    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:51.780781 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:51.780727    2568 patch_prober.go:28] interesting pod/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 18 06:40:51.780992 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:51.780807    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b" podUID="d88eebd0-4865-4103-ab85-92c6c00cbd74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 18 06:40:51.907486 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:51.907448    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:52.331247 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:52.331071    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:52.443691 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:52.443625    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:52.538243 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:52.530735    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:52.780738 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:52.780697    2568 patch_prober.go:28] interesting pod/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 18 06:40:52.781264 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:52.780764    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b" podUID="d88eebd0-4865-4103-ab85-92c6c00cbd74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 18 06:40:53.206080 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:53.205972    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:53.784176 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:53.784147    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:53.784641 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:53.784466    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:53.785105 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:53.785090    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:40:54.023980 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:54.023946    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"]
Sep 18 06:40:54.038292 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:54.031359    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"
Sep 18 06:40:54.042319 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:54.042289    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"]
Sep 18 06:40:54.048410 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:54.048385    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-708b6f-serving-cert\""
Sep 18 06:40:54.071827 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:54.071796    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:54.125188 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:54.125158    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88717d6b-2a8f-4023-a308-0ec483232deb-openshift-service-ca\") pod \"maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf\" (UID: \"88717d6b-2a8f-4023-a308-0ec483232deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"
Sep 18 06:40:54.125457 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:54.125440    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88717d6b-2a8f-4023-a308-0ec483232deb-maas-api-tls\") pod \"maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf\" (UID: \"88717d6b-2a8f-4023-a308-0ec483232deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"
Sep 18 06:40:54.125574 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:54.125563    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blkps\" (UniqueName: \"kubernetes.io/projected/88717d6b-2a8f-4023-a308-0ec483232deb-kube-api-access-blkps\") pod \"maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf\" (UID: \"88717d6b-2a8f-4023-a308-0ec483232deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"
Sep 18 06:40:54.125669 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:54.125659    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88717d6b-2a8f-4023-a308-0ec483232deb-metrics-certs\") pod \"maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf\" (UID: \"88717d6b-2a8f-4023-a308-0ec483232deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"
Sep 18 06:40:54.169948 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:54.169824    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:54.226513 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:54.226482    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88717d6b-2a8f-4023-a308-0ec483232deb-openshift-service-ca\") pod \"maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf\" (UID: \"88717d6b-2a8f-4023-a308-0ec483232deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"
Sep 18 06:40:54.226765 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:54.226749    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88717d6b-2a8f-4023-a308-0ec483232deb-maas-api-tls\") pod \"maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf\" (UID: \"88717d6b-2a8f-4023-a308-0ec483232deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"
Sep 18 06:40:54.226904 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:54.226890    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blkps\" (UniqueName: \"kubernetes.io/projected/88717d6b-2a8f-4023-a308-0ec483232deb-kube-api-access-blkps\") pod \"maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf\" (UID: \"88717d6b-2a8f-4023-a308-0ec483232deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"
Sep 18 06:40:54.227022 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:54.227012    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88717d6b-2a8f-4023-a308-0ec483232deb-metrics-certs\") pod \"maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf\" (UID: \"88717d6b-2a8f-4023-a308-0ec483232deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"
Sep 18 06:40:54.227177 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:40:54.226892    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-serving-cert: secret "maas-api-e2e-inf-b-708b6f-serving-cert" not found
Sep 18 06:40:54.227359 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:54.227064    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88717d6b-2a8f-4023-a308-0ec483232deb-openshift-service-ca\") pod \"maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf\" (UID: \"88717d6b-2a8f-4023-a308-0ec483232deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"
Sep 18 06:40:54.227433 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:40:54.227349    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88717d6b-2a8f-4023-a308-0ec483232deb-maas-api-tls podName:88717d6b-2a8f-4023-a308-0ec483232deb nodeName:}" failed. No retries permitted until 2026-09-18 06:40:54.727327998 +0000 UTC m=+1120.408855728 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/88717d6b-2a8f-4023-a308-0ec483232deb-maas-api-tls") pod "maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf" (UID: "88717d6b-2a8f-4023-a308-0ec483232deb") : secret "maas-api-e2e-inf-b-708b6f-serving-cert" not found
Sep 18 06:40:54.233533 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:54.229615    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88717d6b-2a8f-4023-a308-0ec483232deb-metrics-certs\") pod \"maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf\" (UID: \"88717d6b-2a8f-4023-a308-0ec483232deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"
Sep 18 06:40:54.235002 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:54.234970    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blkps\" (UniqueName: \"kubernetes.io/projected/88717d6b-2a8f-4023-a308-0ec483232deb-kube-api-access-blkps\") pod \"maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf\" (UID: \"88717d6b-2a8f-4023-a308-0ec483232deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"
Sep 18 06:40:54.515475 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:54.515438    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:54.608420 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:54.608391    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:54.755913 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:54.755869    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88717d6b-2a8f-4023-a308-0ec483232deb-maas-api-tls\") pod \"maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf\" (UID: \"88717d6b-2a8f-4023-a308-0ec483232deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"
Sep 18 06:40:54.756478 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:40:54.756454    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-serving-cert: secret "maas-api-e2e-inf-b-708b6f-serving-cert" not found
Sep 18 06:40:54.756820 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:40:54.756807    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88717d6b-2a8f-4023-a308-0ec483232deb-maas-api-tls podName:88717d6b-2a8f-4023-a308-0ec483232deb nodeName:}" failed. No retries permitted until 2026-09-18 06:40:55.756673696 +0000 UTC m=+1121.438201421 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/88717d6b-2a8f-4023-a308-0ec483232deb-maas-api-tls") pod "maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf" (UID: "88717d6b-2a8f-4023-a308-0ec483232deb") : secret "maas-api-e2e-inf-b-708b6f-serving-cert" not found
Sep 18 06:40:54.837108 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:54.837020    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:54.938986 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:54.938919    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:55.040057 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:55.040019    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:55.742351 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:55.742315    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:55.769443 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:55.769402    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88717d6b-2a8f-4023-a308-0ec483232deb-maas-api-tls\") pod \"maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf\" (UID: \"88717d6b-2a8f-4023-a308-0ec483232deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"
Sep 18 06:40:55.772976 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:55.772941    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88717d6b-2a8f-4023-a308-0ec483232deb-maas-api-tls\") pod \"maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf\" (UID: \"88717d6b-2a8f-4023-a308-0ec483232deb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"
Sep 18 06:40:55.867630 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:55.867592    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"
Sep 18 06:40:56.152641 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:56.151209    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"]
Sep 18 06:40:56.154702 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:40:56.154665    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88717d6b_2a8f_4023_a308_0ec483232deb.slice/crio-82cd34cc552f691fc42f916fe7b73e462cdaca37a54986cfdfcab27c8da687e9 WatchSource:0}: Error finding container 82cd34cc552f691fc42f916fe7b73e462cdaca37a54986cfdfcab27c8da687e9: Status 404 returned error can't find the container with id 82cd34cc552f691fc42f916fe7b73e462cdaca37a54986cfdfcab27c8da687e9
Sep 18 06:40:56.359997 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:56.359903    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf" event={"ID":"88717d6b-2a8f-4023-a308-0ec483232deb","Type":"ContainerStarted","Data":"82cd34cc552f691fc42f916fe7b73e462cdaca37a54986cfdfcab27c8da687e9"}
Sep 18 06:40:56.362064 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:56.361983    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:57.039041 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:57.039001    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:57.368355 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:57.368268    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf" event={"ID":"88717d6b-2a8f-4023-a308-0ec483232deb","Type":"ContainerStarted","Data":"2ad9387f582d5f29a123085b2fd65ba18d05ed36bf091cf311d5af01d8bad82a"}
Sep 18 06:40:57.368503 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:57.368353    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"
Sep 18 06:40:57.384672 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:57.384626    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf" podStartSLOduration=2.971100003 podStartE2EDuration="3.384608947s" totalImagesPullingTime="413.508944ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:40:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:40:57.382990928 +0000 UTC m=+1123.064518671" watchObservedRunningTime="2026-09-18 06:40:57.384608947 +0000 UTC m=+1123.066136691"
Sep 18 06:40:57.633343 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:57.633255    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:40:59.326160 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:59.326120    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:59.326566 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:59.326181    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:59.366399 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:59.366366    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:40:59.390913 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:40:59.390884    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:41:00.960554 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:00.960521    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:01.391146 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:01.391070    2568 generic.go:356] "Generic (PLEG): container finished" podID="0f6c5262-48e2-42b1-9e20-45fe353bc292" containerID="3e8a27f6d405745b8130f7951277864691983c05e151a19a4c0eb8f439eb7f5c" exitCode=0
Sep 18 06:41:01.391317 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:01.391143    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts" event={"ID":"0f6c5262-48e2-42b1-9e20-45fe353bc292","Type":"ContainerDied","Data":"3e8a27f6d405745b8130f7951277864691983c05e151a19a4c0eb8f439eb7f5c"}
Sep 18 06:41:01.391626 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:01.391604    2568 scope.go:117] "RemoveContainer" containerID="3e8a27f6d405745b8130f7951277864691983c05e151a19a4c0eb8f439eb7f5c"
Sep 18 06:41:01.392828 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:01.392798    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:41:01.392935 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:01.392883    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:41:01.392993 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:01.392931    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:41:01.509497 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:01.509423    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:01.572245 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:01.572178    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:01.657093 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:01.657035    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:01.766705 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:01.766672    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:01.798096 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:01.798052    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"]
Sep 18 06:41:01.798681 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:01.798641    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd" podUID="3764aa9c-4226-414b-a644-01a8e3b4b0a8" containerName="istio-proxy" containerID="cri-o://56a31fcbcf34e4e9bd74c8cd49b511347fdc6b8726faee529579ea11a3b5c408" gracePeriod=30
Sep 18 06:41:01.833237 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:01.833157    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:02.132879 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:02.132789    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:02.249699 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:02.249671    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:02.348725 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:02.348683    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:02.399859 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:02.399824    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts" event={"ID":"0f6c5262-48e2-42b1-9e20-45fe353bc292","Type":"ContainerStarted","Data":"cda555119626fa286662a4162b25e4a6a3198564165791f3eeca9fd6afa48dfa"}
Sep 18 06:41:02.456944 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:02.456913    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:02.761676 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:02.761553    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:02.923923 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:02.923882    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:41:02.925115 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:02.925085    2568 patch_prober.go:28] interesting pod/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 18 06:41:02.925272 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:02.925151    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts" podUID="0f6c5262-48e2-42b1-9e20-45fe353bc292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 18 06:41:02.945564 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:02.945457    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:03.042810 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:03.042622    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:03.380499 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:03.380430    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"
Sep 18 06:41:03.844289 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:03.842099    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:03.924202 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:03.923820    2568 patch_prober.go:28] interesting pod/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 18 06:41:03.924202 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:03.923883    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts" podUID="0f6c5262-48e2-42b1-9e20-45fe353bc292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 18 06:41:03.994682 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:03.994660    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:41:04.071755 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.071719    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m44g4\" (UniqueName: \"kubernetes.io/projected/3764aa9c-4226-414b-a644-01a8e3b4b0a8-kube-api-access-m44g4\") pod \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") "
Sep 18 06:41:04.071930 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.071774    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-token\") pod \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") "
Sep 18 06:41:04.071930 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.071814    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-credential-socket\") pod \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") "
Sep 18 06:41:04.071930 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.071889    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-podinfo\") pod \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") "
Sep 18 06:41:04.072081 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.071930    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-envoy\") pod \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") "
Sep 18 06:41:04.072081 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.071968    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-workload-socket\") pod \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") "
Sep 18 06:41:04.072081 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.072021    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-workload-certs\") pod \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") "
Sep 18 06:41:04.072081 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.072056    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-data\") pod \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") "
Sep 18 06:41:04.072352 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.072086    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istiod-ca-cert\") pod \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\" (UID: \"3764aa9c-4226-414b-a644-01a8e3b4b0a8\") "
Sep 18 06:41:04.072352 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.072208    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3764aa9c-4226-414b-a644-01a8e3b4b0a8" (UID: "3764aa9c-4226-414b-a644-01a8e3b4b0a8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:41:04.072352 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.072258    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3764aa9c-4226-414b-a644-01a8e3b4b0a8" (UID: "3764aa9c-4226-414b-a644-01a8e3b4b0a8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:41:04.072518 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.072451    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3764aa9c-4226-414b-a644-01a8e3b4b0a8" (UID: "3764aa9c-4226-414b-a644-01a8e3b4b0a8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:41:04.072518 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.072464    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-data" (OuterVolumeSpecName: "istio-data") pod "3764aa9c-4226-414b-a644-01a8e3b4b0a8" (UID: "3764aa9c-4226-414b-a644-01a8e3b4b0a8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:41:04.072637 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.072596    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-credential-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:41:04.072637 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.072621    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-workload-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:41:04.072637 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.072635    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-workload-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:41:04.072798 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.072648    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-data\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:41:04.072798 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.072748    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3764aa9c-4226-414b-a644-01a8e3b4b0a8" (UID: "3764aa9c-4226-414b-a644-01a8e3b4b0a8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 06:41:04.074031 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.073994    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-token" (OuterVolumeSpecName: "istio-token") pod "3764aa9c-4226-414b-a644-01a8e3b4b0a8" (UID: "3764aa9c-4226-414b-a644-01a8e3b4b0a8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:41:04.074416 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.074394    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3764aa9c-4226-414b-a644-01a8e3b4b0a8" (UID: "3764aa9c-4226-414b-a644-01a8e3b4b0a8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 06:41:04.074600 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.074582    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3764aa9c-4226-414b-a644-01a8e3b4b0a8-kube-api-access-m44g4" (OuterVolumeSpecName: "kube-api-access-m44g4") pod "3764aa9c-4226-414b-a644-01a8e3b4b0a8" (UID: "3764aa9c-4226-414b-a644-01a8e3b4b0a8"). InnerVolumeSpecName "kube-api-access-m44g4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:41:04.074926 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.074903    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3764aa9c-4226-414b-a644-01a8e3b4b0a8" (UID: "3764aa9c-4226-414b-a644-01a8e3b4b0a8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:41:04.173400 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.173368    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m44g4\" (UniqueName: \"kubernetes.io/projected/3764aa9c-4226-414b-a644-01a8e3b4b0a8-kube-api-access-m44g4\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:41:04.173400 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.173394    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-token\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:41:04.173400 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.173404    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-podinfo\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:41:04.173400 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.173413    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istio-envoy\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:41:04.173671 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.173421    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3764aa9c-4226-414b-a644-01a8e3b4b0a8-istiod-ca-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:41:04.207130 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.207103    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:41:04.413074 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.413044    2568 generic.go:356] "Generic (PLEG): container finished" podID="3764aa9c-4226-414b-a644-01a8e3b4b0a8" containerID="56a31fcbcf34e4e9bd74c8cd49b511347fdc6b8726faee529579ea11a3b5c408" exitCode=0
Sep 18 06:41:04.413546 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.413093    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"
Sep 18 06:41:04.413546 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.413129    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd" event={"ID":"3764aa9c-4226-414b-a644-01a8e3b4b0a8","Type":"ContainerDied","Data":"56a31fcbcf34e4e9bd74c8cd49b511347fdc6b8726faee529579ea11a3b5c408"}
Sep 18 06:41:04.413546 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.413182    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd" event={"ID":"3764aa9c-4226-414b-a644-01a8e3b4b0a8","Type":"ContainerDied","Data":"a7a3b227094e3194c0b9a2fe0816d98c4e73c7c3cdb81de0b87fbbeafb69dc44"}
Sep 18 06:41:04.413546 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.413204    2568 scope.go:117] "RemoveContainer" containerID="56a31fcbcf34e4e9bd74c8cd49b511347fdc6b8726faee529579ea11a3b5c408"
Sep 18 06:41:04.422364 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.422284    2568 scope.go:117] "RemoveContainer" containerID="56a31fcbcf34e4e9bd74c8cd49b511347fdc6b8726faee529579ea11a3b5c408"
Sep 18 06:41:04.422565 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:41:04.422547    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56a31fcbcf34e4e9bd74c8cd49b511347fdc6b8726faee529579ea11a3b5c408\": container with ID starting with 56a31fcbcf34e4e9bd74c8cd49b511347fdc6b8726faee529579ea11a3b5c408 not found: ID does not exist" containerID="56a31fcbcf34e4e9bd74c8cd49b511347fdc6b8726faee529579ea11a3b5c408"
Sep 18 06:41:04.422624 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.422574    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56a31fcbcf34e4e9bd74c8cd49b511347fdc6b8726faee529579ea11a3b5c408"} err="failed to get container status \"56a31fcbcf34e4e9bd74c8cd49b511347fdc6b8726faee529579ea11a3b5c408\": rpc error: code = NotFound desc = could not find container \"56a31fcbcf34e4e9bd74c8cd49b511347fdc6b8726faee529579ea11a3b5c408\": container with ID starting with 56a31fcbcf34e4e9bd74c8cd49b511347fdc6b8726faee529579ea11a3b5c408 not found: ID does not exist"
Sep 18 06:41:04.441805 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.441775    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"]
Sep 18 06:41:04.445106 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.445080    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd"]
Sep 18 06:41:04.548279 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.548241    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3764aa9c-4226-414b-a644-01a8e3b4b0a8" path="/var/lib/kubelet/pods/3764aa9c-4226-414b-a644-01a8e3b4b0a8/volumes"
Sep 18 06:41:04.725288 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.725180    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:04.923992 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.923960    2568 patch_prober.go:28] interesting pod/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 18 06:41:04.924156 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:04.924015    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts" podUID="0f6c5262-48e2-42b1-9e20-45fe353bc292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 18 06:41:05.235630 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:05.235591    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:05.336454 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:05.336415    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:05.600045 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:05.599974    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:05.923661 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:05.923629    2568 patch_prober.go:28] interesting pod/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 18 06:41:05.923831 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:05.923694    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts" podUID="0f6c5262-48e2-42b1-9e20-45fe353bc292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 18 06:41:06.337995 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:06.337765    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:06.923847 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:06.923815    2568 patch_prober.go:28] interesting pod/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 18 06:41:06.924280 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:06.923872    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts" podUID="0f6c5262-48e2-42b1-9e20-45fe353bc292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 18 06:41:06.997019 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:06.996981    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:07.089080 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:07.089037    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:07.359528 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:07.359408    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:07.487522 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:07.487473    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:07.585240 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:07.585036    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:07.923608 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:07.923572    2568 patch_prober.go:28] interesting pod/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 18 06:41:07.923812 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:07.923666    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts" podUID="0f6c5262-48e2-42b1-9e20-45fe353bc292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 18 06:41:08.023206 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:08.023175    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:08.735838 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:08.735782    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:08.928458 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:08.928429    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:41:08.929704 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:08.929680    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:41:09.491675 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:09.491638    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:09.999515 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:09.999481    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:10.109152 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:10.109114    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:10.754172 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:10.754109    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:11.479213 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:11.479183    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:11.570728 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:11.570691    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:11.722339 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:11.722314    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:12.163376 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:12.163346    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:12.813199 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:12.813164    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:13.519618 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:13.519582    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:14.144802 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:14.144772    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:15.485119 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:15.485088    2568 generic.go:356] "Generic (PLEG): container finished" podID="96c8cb03-28cf-41cb-829d-a7b9073bdb1c" containerID="b3266bfbf38d05db250319aa0dd14a0a8ce457454a807ad0a57a8b316afca9e6" exitCode=0
Sep 18 06:41:15.485562 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:15.485175    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b" event={"ID":"96c8cb03-28cf-41cb-829d-a7b9073bdb1c","Type":"ContainerDied","Data":"b3266bfbf38d05db250319aa0dd14a0a8ce457454a807ad0a57a8b316afca9e6"}
Sep 18 06:41:15.485635 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:15.485621    2568 scope.go:117] "RemoveContainer" containerID="b3266bfbf38d05db250319aa0dd14a0a8ce457454a807ad0a57a8b316afca9e6"
Sep 18 06:41:15.486915 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:15.486864    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:41:15.486995 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:15.486932    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:41:15.486995 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:15.486970    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:41:16.491796 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:16.491768    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b" event={"ID":"96c8cb03-28cf-41cb-829d-a7b9073bdb1c","Type":"ContainerStarted","Data":"9e7bccb2d87e1176789e1b18af025644227fdf29474633d9914d34a16d3c13c3"}
Sep 18 06:41:17.041171 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:17.041133    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:41:17.042347 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:17.042325    2568 patch_prober.go:28] interesting pod/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 18 06:41:17.042420 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:17.042377    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b" podUID="96c8cb03-28cf-41cb-829d-a7b9073bdb1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 18 06:41:17.227470 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:17.227435    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:17.291032 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:17.290996    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:17.364596 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:17.364484    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:17.463368 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:17.463312    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:17.562752 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:17.562711    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:17.800694 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:17.800661    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:41:18.041081 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:18.041036    2568 patch_prober.go:28] interesting pod/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 18 06:41:18.041426 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:18.041109    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b" podUID="96c8cb03-28cf-41cb-829d-a7b9073bdb1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 18 06:41:18.465351 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:18.465317    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:19.040987 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:19.040948    2568 patch_prober.go:28] interesting pod/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 18 06:41:19.041627 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:19.041592    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b" podUID="96c8cb03-28cf-41cb-829d-a7b9073bdb1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 18 06:41:19.303955 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:19.303881    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:41:20.040772 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:20.040724    2568 patch_prober.go:28] interesting pod/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 18 06:41:20.041010 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:20.040797    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b" podUID="96c8cb03-28cf-41cb-829d-a7b9073bdb1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 18 06:41:21.041138 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:21.041090    2568 patch_prober.go:28] interesting pod/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 18 06:41:21.041668 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:21.041161    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b" podUID="96c8cb03-28cf-41cb-829d-a7b9073bdb1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 18 06:41:22.040717 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:22.040677    2568 patch_prober.go:28] interesting pod/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 18 06:41:22.041037 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:22.041007    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b" podUID="96c8cb03-28cf-41cb-829d-a7b9073bdb1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 18 06:41:23.044297 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:23.044268    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:41:23.045232 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:23.045197    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:41:23.126867 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:23.126830    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:23.195925 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:23.195898    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:31.471869 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:31.471788    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:32.822521 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:32.822482    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:33.636078 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:33.636045    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:34.246656 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:34.246625    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:34.932978 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:34.932940    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:35.693711 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:35.693672    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:36.394131 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:36.394097    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:37.078419 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:37.078392    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:37.138330 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:37.138299    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:37.810552 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:37.810508    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:38.525147 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:38.525113    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:39.132753 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:39.132722    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:39.759981 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:39.759945    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:46.517461 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:46.517426    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:47.058668 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:47.058638    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:47.126053 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:47.126017    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:47.226334 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:47.226298    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:47.283667 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:47.283632    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:47.384761 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:47.384672    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:47.701995 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:47.701960    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:47.810678 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:47.810612    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:48.111670 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:48.111581    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:48.208371 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:48.208323    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:48.424401 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:48.424366    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:49.343097 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:49.343050    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:50.129550 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:50.129514    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:50.249050 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:50.249004    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:50.345071 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:50.345025    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:50.882993 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:50.882962    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:51.647462 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:51.647430    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:52.349631 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:52.349597    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"]
Sep 18 06:41:52.350112 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:52.350093    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3764aa9c-4226-414b-a644-01a8e3b4b0a8" containerName="istio-proxy"
Sep 18 06:41:52.350187 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:52.350116    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3764aa9c-4226-414b-a644-01a8e3b4b0a8" containerName="istio-proxy"
Sep 18 06:41:52.350265 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:52.350229    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3764aa9c-4226-414b-a644-01a8e3b4b0a8" containerName="istio-proxy"
Sep 18 06:41:52.353564 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:52.353548    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"
Sep 18 06:41:52.355888 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:52.355869    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-5e4c6960-serving-cert\""
Sep 18 06:41:52.363515 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:52.363491    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"]
Sep 18 06:41:52.398347 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:52.398322    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:52.487493 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:52.487448    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9566255-6f71-4f3d-a471-8c208dc32a7c-openshift-service-ca\") pod \"maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq\" (UID: \"d9566255-6f71-4f3d-a471-8c208dc32a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"
Sep 18 06:41:52.487692 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:52.487573    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9566255-6f71-4f3d-a471-8c208dc32a7c-maas-api-tls\") pod \"maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq\" (UID: \"d9566255-6f71-4f3d-a471-8c208dc32a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"
Sep 18 06:41:52.487692 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:52.487607    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9566255-6f71-4f3d-a471-8c208dc32a7c-metrics-certs\") pod \"maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq\" (UID: \"d9566255-6f71-4f3d-a471-8c208dc32a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"
Sep 18 06:41:52.487692 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:52.487635    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6wpm\" (UniqueName: \"kubernetes.io/projected/d9566255-6f71-4f3d-a471-8c208dc32a7c-kube-api-access-s6wpm\") pod \"maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq\" (UID: \"d9566255-6f71-4f3d-a471-8c208dc32a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"
Sep 18 06:41:52.498897 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:52.498856    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:52.588703 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:52.588673    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9566255-6f71-4f3d-a471-8c208dc32a7c-openshift-service-ca\") pod \"maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq\" (UID: \"d9566255-6f71-4f3d-a471-8c208dc32a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"
Sep 18 06:41:52.588877 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:52.588740    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9566255-6f71-4f3d-a471-8c208dc32a7c-maas-api-tls\") pod \"maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq\" (UID: \"d9566255-6f71-4f3d-a471-8c208dc32a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"
Sep 18 06:41:52.588877 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:52.588764    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9566255-6f71-4f3d-a471-8c208dc32a7c-metrics-certs\") pod \"maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq\" (UID: \"d9566255-6f71-4f3d-a471-8c208dc32a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"
Sep 18 06:41:52.588877 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:52.588785    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6wpm\" (UniqueName: \"kubernetes.io/projected/d9566255-6f71-4f3d-a471-8c208dc32a7c-kube-api-access-s6wpm\") pod \"maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq\" (UID: \"d9566255-6f71-4f3d-a471-8c208dc32a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"
Sep 18 06:41:52.589018 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:41:52.588890    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-serving-cert: secret "maas-api-e2e-ait-5e4c6960-serving-cert" not found
Sep 18 06:41:52.589018 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:41:52.588965    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9566255-6f71-4f3d-a471-8c208dc32a7c-maas-api-tls podName:d9566255-6f71-4f3d-a471-8c208dc32a7c nodeName:}" failed. No retries permitted until 2026-09-18 06:41:53.088944803 +0000 UTC m=+1178.770472541 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d9566255-6f71-4f3d-a471-8c208dc32a7c-maas-api-tls") pod "maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq" (UID: "d9566255-6f71-4f3d-a471-8c208dc32a7c") : secret "maas-api-e2e-ait-5e4c6960-serving-cert" not found
Sep 18 06:41:52.589401 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:52.589377    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9566255-6f71-4f3d-a471-8c208dc32a7c-openshift-service-ca\") pod \"maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq\" (UID: \"d9566255-6f71-4f3d-a471-8c208dc32a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"
Sep 18 06:41:52.591156 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:52.591135    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9566255-6f71-4f3d-a471-8c208dc32a7c-metrics-certs\") pod \"maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq\" (UID: \"d9566255-6f71-4f3d-a471-8c208dc32a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"
Sep 18 06:41:52.599040 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:52.599015    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6wpm\" (UniqueName: \"kubernetes.io/projected/d9566255-6f71-4f3d-a471-8c208dc32a7c-kube-api-access-s6wpm\") pod \"maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq\" (UID: \"d9566255-6f71-4f3d-a471-8c208dc32a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"
Sep 18 06:41:52.845340 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:52.845305    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:52.919304 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:52.919272    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:53.012877 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:53.012846    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:53.096367 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:53.096270    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9566255-6f71-4f3d-a471-8c208dc32a7c-maas-api-tls\") pod \"maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq\" (UID: \"d9566255-6f71-4f3d-a471-8c208dc32a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"
Sep 18 06:41:53.099342 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:53.099301    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9566255-6f71-4f3d-a471-8c208dc32a7c-maas-api-tls\") pod \"maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq\" (UID: \"d9566255-6f71-4f3d-a471-8c208dc32a7c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"
Sep 18 06:41:53.265124 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:53.265087    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"
Sep 18 06:41:53.323915 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:53.323880    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:53.537343 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:53.537319    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"]
Sep 18 06:41:53.539608 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:41:53.539585    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9566255_6f71_4f3d_a471_8c208dc32a7c.slice/crio-5f792b09ca3bea12bc375639cf2a027222784082e4b2e6dc88a35d1db1ea8c86 WatchSource:0}: Error finding container 5f792b09ca3bea12bc375639cf2a027222784082e4b2e6dc88a35d1db1ea8c86: Status 404 returned error can't find the container with id 5f792b09ca3bea12bc375639cf2a027222784082e4b2e6dc88a35d1db1ea8c86
Sep 18 06:41:53.608649 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:53.608616    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:41:53.688436 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:53.688398    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq" event={"ID":"d9566255-6f71-4f3d-a471-8c208dc32a7c","Type":"ContainerStarted","Data":"5f792b09ca3bea12bc375639cf2a027222784082e4b2e6dc88a35d1db1ea8c86"}
Sep 18 06:41:54.027881 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:54.027854    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:54.128477 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:54.128450    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:54.646011 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:54.645981    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:41:54.694781 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:54.694741    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq" event={"ID":"d9566255-6f71-4f3d-a471-8c208dc32a7c","Type":"ContainerStarted","Data":"d5f7502eb5b09acd66a746f1850bd3d27cdb203db12d79cacb658233b09cca81"}
Sep 18 06:41:54.694966 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:54.694880    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"
Sep 18 06:41:54.712448 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:54.712392    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq" podStartSLOduration=2.335335381 podStartE2EDuration="2.712371608s" totalImagesPullingTime="377.036227ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:41:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:41:54.709757008 +0000 UTC m=+1180.391284762" watchObservedRunningTime="2026-09-18 06:41:54.712371608 +0000 UTC m=+1180.393899352"
Sep 18 06:41:55.394365 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:41:55.394336    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:00.705741 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:00.705710    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"
Sep 18 06:42:09.383550 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:09.383518    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:42:17.159173 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:17.159103    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:17.792594 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:17.792546    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:18.517062 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:18.517020    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:19.120244 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:19.120191    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:19.743910 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:19.743870    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:20.530621 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:20.530583    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:21.311086 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:21.311055    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:21.921493 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:21.921451    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:22.624790 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:22.624762    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:23.271098 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:23.271062    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:23.905841 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:23.905802    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:24.006193 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:24.006160    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:24.628277 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:24.628243    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:25.242213 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:25.242177    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:25.892166 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:25.892131    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:28.682334 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.682298    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"]
Sep 18 06:42:28.686124 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.686102    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:28.688618 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.688599    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-946609\"/\"default-dockercfg-sbzpb\""
Sep 18 06:42:28.688702 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.688672    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-946609\"/\"kube-root-ca.crt\""
Sep 18 06:42:28.688752 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.688722    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-946609\"/\"openshift-service-ca.crt\""
Sep 18 06:42:28.689741 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.689723    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-946609\"/\"simulator-w0-946609-kserve-self-signed-certs\""
Sep 18 06:42:28.694950 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.694928    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"]
Sep 18 06:42:28.781861 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.781832    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:28.841176 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.841143    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgzhr\" (UniqueName: \"kubernetes.io/projected/f64d9a93-2a33-412b-b470-77e030c9b071-kube-api-access-vgzhr\") pod \"simulator-w0-946609-kserve-67569d6bcf-8n4m4\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") " pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:28.841331 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.841188    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-tmp-dir\") pod \"simulator-w0-946609-kserve-67569d6bcf-8n4m4\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") " pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:28.841331 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.841284    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-model-cache\") pod \"simulator-w0-946609-kserve-67569d6bcf-8n4m4\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") " pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:28.841405 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.841362    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-home\") pod \"simulator-w0-946609-kserve-67569d6bcf-8n4m4\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") " pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:28.841444 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.841421    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f64d9a93-2a33-412b-b470-77e030c9b071-tls-certs\") pod \"simulator-w0-946609-kserve-67569d6bcf-8n4m4\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") " pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:28.841480 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.841465    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-dshm\") pod \"simulator-w0-946609-kserve-67569d6bcf-8n4m4\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") " pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:28.942199 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.942117    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f64d9a93-2a33-412b-b470-77e030c9b071-tls-certs\") pod \"simulator-w0-946609-kserve-67569d6bcf-8n4m4\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") " pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:28.942199 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.942170    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-dshm\") pod \"simulator-w0-946609-kserve-67569d6bcf-8n4m4\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") " pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:28.942443 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.942258    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgzhr\" (UniqueName: \"kubernetes.io/projected/f64d9a93-2a33-412b-b470-77e030c9b071-kube-api-access-vgzhr\") pod \"simulator-w0-946609-kserve-67569d6bcf-8n4m4\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") " pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:28.942443 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.942384    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-tmp-dir\") pod \"simulator-w0-946609-kserve-67569d6bcf-8n4m4\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") " pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:28.942563 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.942449    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-model-cache\") pod \"simulator-w0-946609-kserve-67569d6bcf-8n4m4\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") " pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:28.942563 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.942522    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-home\") pod \"simulator-w0-946609-kserve-67569d6bcf-8n4m4\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") " pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:28.942842 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.942814    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-tmp-dir\") pod \"simulator-w0-946609-kserve-67569d6bcf-8n4m4\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") " pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:28.942956 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.942922    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-home\") pod \"simulator-w0-946609-kserve-67569d6bcf-8n4m4\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") " pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:28.942956 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.942927    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-model-cache\") pod \"simulator-w0-946609-kserve-67569d6bcf-8n4m4\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") " pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:28.944586 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.944552    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-dshm\") pod \"simulator-w0-946609-kserve-67569d6bcf-8n4m4\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") " pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:28.944920 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.944899    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f64d9a93-2a33-412b-b470-77e030c9b071-tls-certs\") pod \"simulator-w0-946609-kserve-67569d6bcf-8n4m4\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") " pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:28.950289 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.950262    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgzhr\" (UniqueName: \"kubernetes.io/projected/f64d9a93-2a33-412b-b470-77e030c9b071-kube-api-access-vgzhr\") pod \"simulator-w0-946609-kserve-67569d6bcf-8n4m4\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") " pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:28.996177 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:28.996154    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:29.270398 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:29.270312    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"]
Sep 18 06:42:29.273843 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:42:29.273808    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf64d9a93_2a33_412b_b470_77e030c9b071.slice/crio-af0b5bc468133df06e5389aa423321919ca4214e175e0acc50a8509016711957 WatchSource:0}: Error finding container af0b5bc468133df06e5389aa423321919ca4214e175e0acc50a8509016711957: Status 404 returned error can't find the container with id af0b5bc468133df06e5389aa423321919ca4214e175e0acc50a8509016711957
Sep 18 06:42:29.880128 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:29.880096    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4" event={"ID":"f64d9a93-2a33-412b-b470-77e030c9b071","Type":"ContainerStarted","Data":"9c7688b8bf871f8fa93e78d13d75e5e96427e6cdd1316643a0a304c295be23fd"}
Sep 18 06:42:29.880531 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:29.880137    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4" event={"ID":"f64d9a93-2a33-412b-b470-77e030c9b071","Type":"ContainerStarted","Data":"af0b5bc468133df06e5389aa423321919ca4214e175e0acc50a8509016711957"}
Sep 18 06:42:29.897610 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:29.897559    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4" podStartSLOduration=1.897544071 podStartE2EDuration="1.897544071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:42:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:42:29.89610052 +0000 UTC m=+1215.577628266" watchObservedRunningTime="2026-09-18 06:42:29.897544071 +0000 UTC m=+1215.579071814"
Sep 18 06:42:31.931958 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:31.931922    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"]
Sep 18 06:42:31.932427 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:31.932165    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq" podUID="d9566255-6f71-4f3d-a471-8c208dc32a7c" containerName="maas-api" containerID="cri-o://d5f7502eb5b09acd66a746f1850bd3d27cdb203db12d79cacb658233b09cca81" gracePeriod=30
Sep 18 06:42:31.953901 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:31.953865    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:32.194706 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:32.194645    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"
Sep 18 06:42:32.276175 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:32.276139    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9566255-6f71-4f3d-a471-8c208dc32a7c-maas-api-tls\") pod \"d9566255-6f71-4f3d-a471-8c208dc32a7c\" (UID: \"d9566255-6f71-4f3d-a471-8c208dc32a7c\") "
Sep 18 06:42:32.276375 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:32.276251    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9566255-6f71-4f3d-a471-8c208dc32a7c-metrics-certs\") pod \"d9566255-6f71-4f3d-a471-8c208dc32a7c\" (UID: \"d9566255-6f71-4f3d-a471-8c208dc32a7c\") "
Sep 18 06:42:32.276375 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:32.276309    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6wpm\" (UniqueName: \"kubernetes.io/projected/d9566255-6f71-4f3d-a471-8c208dc32a7c-kube-api-access-s6wpm\") pod \"d9566255-6f71-4f3d-a471-8c208dc32a7c\" (UID: \"d9566255-6f71-4f3d-a471-8c208dc32a7c\") "
Sep 18 06:42:32.276375 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:32.276351    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9566255-6f71-4f3d-a471-8c208dc32a7c-openshift-service-ca\") pod \"d9566255-6f71-4f3d-a471-8c208dc32a7c\" (UID: \"d9566255-6f71-4f3d-a471-8c208dc32a7c\") "
Sep 18 06:42:32.276899 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:32.276868    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9566255-6f71-4f3d-a471-8c208dc32a7c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d9566255-6f71-4f3d-a471-8c208dc32a7c" (UID: "d9566255-6f71-4f3d-a471-8c208dc32a7c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 06:42:32.278683 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:32.278646    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9566255-6f71-4f3d-a471-8c208dc32a7c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d9566255-6f71-4f3d-a471-8c208dc32a7c" (UID: "d9566255-6f71-4f3d-a471-8c208dc32a7c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:42:32.278790 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:32.278714    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9566255-6f71-4f3d-a471-8c208dc32a7c-kube-api-access-s6wpm" (OuterVolumeSpecName: "kube-api-access-s6wpm") pod "d9566255-6f71-4f3d-a471-8c208dc32a7c" (UID: "d9566255-6f71-4f3d-a471-8c208dc32a7c"). InnerVolumeSpecName "kube-api-access-s6wpm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:42:32.278790 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:32.278740    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9566255-6f71-4f3d-a471-8c208dc32a7c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d9566255-6f71-4f3d-a471-8c208dc32a7c" (UID: "d9566255-6f71-4f3d-a471-8c208dc32a7c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:42:32.377650 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:32.377617    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9566255-6f71-4f3d-a471-8c208dc32a7c-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:42:32.377650 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:32.377643    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d9566255-6f71-4f3d-a471-8c208dc32a7c-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:42:32.377650 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:32.377653    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6wpm\" (UniqueName: \"kubernetes.io/projected/d9566255-6f71-4f3d-a471-8c208dc32a7c-kube-api-access-s6wpm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:42:32.377877 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:32.377663    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9566255-6f71-4f3d-a471-8c208dc32a7c-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:42:32.897176 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:32.897148    2568 generic.go:356] "Generic (PLEG): container finished" podID="d9566255-6f71-4f3d-a471-8c208dc32a7c" containerID="d5f7502eb5b09acd66a746f1850bd3d27cdb203db12d79cacb658233b09cca81" exitCode=0
Sep 18 06:42:32.897347 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:32.897191    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"
Sep 18 06:42:32.897347 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:32.897249    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq" event={"ID":"d9566255-6f71-4f3d-a471-8c208dc32a7c","Type":"ContainerDied","Data":"d5f7502eb5b09acd66a746f1850bd3d27cdb203db12d79cacb658233b09cca81"}
Sep 18 06:42:32.897347 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:32.897301    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq" event={"ID":"d9566255-6f71-4f3d-a471-8c208dc32a7c","Type":"ContainerDied","Data":"5f792b09ca3bea12bc375639cf2a027222784082e4b2e6dc88a35d1db1ea8c86"}
Sep 18 06:42:32.897347 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:32.897324    2568 scope.go:117] "RemoveContainer" containerID="d5f7502eb5b09acd66a746f1850bd3d27cdb203db12d79cacb658233b09cca81"
Sep 18 06:42:32.906113 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:32.906090    2568 scope.go:117] "RemoveContainer" containerID="d5f7502eb5b09acd66a746f1850bd3d27cdb203db12d79cacb658233b09cca81"
Sep 18 06:42:32.906452 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:42:32.906424    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5f7502eb5b09acd66a746f1850bd3d27cdb203db12d79cacb658233b09cca81\": container with ID starting with d5f7502eb5b09acd66a746f1850bd3d27cdb203db12d79cacb658233b09cca81 not found: ID does not exist" containerID="d5f7502eb5b09acd66a746f1850bd3d27cdb203db12d79cacb658233b09cca81"
Sep 18 06:42:32.906578 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:32.906460    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5f7502eb5b09acd66a746f1850bd3d27cdb203db12d79cacb658233b09cca81"} err="failed to get container status \"d5f7502eb5b09acd66a746f1850bd3d27cdb203db12d79cacb658233b09cca81\": rpc error: code = NotFound desc = could not find container \"d5f7502eb5b09acd66a746f1850bd3d27cdb203db12d79cacb658233b09cca81\": container with ID starting with d5f7502eb5b09acd66a746f1850bd3d27cdb203db12d79cacb658233b09cca81 not found: ID does not exist"
Sep 18 06:42:32.913183 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:32.913162    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"]
Sep 18 06:42:32.916972 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:32.916947    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5e4c6960-647fdb5bf6-lzrjq"]
Sep 18 06:42:33.191048 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:33.190962    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:33.247318 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:33.247284    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:33.321962 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:33.321933    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:33.616437 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:33.616358    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:33.905984 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:33.905942    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:34.005791 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:34.005757    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:34.247398 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:34.247295    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:34.543787 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:34.543713    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9566255-6f71-4f3d-a471-8c208dc32a7c" path="/var/lib/kubelet/pods/d9566255-6f71-4f3d-a471-8c208dc32a7c/volumes"
Sep 18 06:42:34.840953 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:34.840816    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:35.466750 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:35.466722    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:35.788890 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:35.788811    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:42:36.281492 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:36.281462    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:36.646130 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:36.646013    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:36.744746 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:36.744718    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:37.161555 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:37.161525    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:37.824511 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:37.824479    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:38.531603 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:38.531561    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:38.996655 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:38.996620    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:38.997115 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:38.996674    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:39.009339 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:39.009317    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:39.233539 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:39.233510    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:39.858555 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:39.858521    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:39.951101 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:39.951070    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:42:40.476907 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:40.476845    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:40.531829 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:40.531784    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:40.724368 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:40.724336    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:40.984641 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:40.984592    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:41.045800 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:41.045767    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:51.576424 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.576388    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"]
Sep 18 06:42:51.576925 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.576890    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9566255-6f71-4f3d-a471-8c208dc32a7c" containerName="maas-api"
Sep 18 06:42:51.576925 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.576906    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9566255-6f71-4f3d-a471-8c208dc32a7c" containerName="maas-api"
Sep 18 06:42:51.577055 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.576992    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9566255-6f71-4f3d-a471-8c208dc32a7c" containerName="maas-api"
Sep 18 06:42:51.580336 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.580318    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:42:51.582683 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.582660    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-43fdfe\"/\"default-dockercfg-f4k6z\""
Sep 18 06:42:51.583689 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.583665    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-43fdfe\"/\"kube-root-ca.crt\""
Sep 18 06:42:51.583814 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.583754    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-43fdfe\"/\"autores-model-43fdfe-kserve-self-signed-certs\""
Sep 18 06:42:51.583814 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.583771    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-43fdfe\"/\"openshift-service-ca.crt\""
Sep 18 06:42:51.589559 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.589536    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"]
Sep 18 06:42:51.670758 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.670729    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-model-cache\") pod \"autores-model-43fdfe-kserve-65f6476d47-wq2jt\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") " pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:42:51.670905 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.670771    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-dshm\") pod \"autores-model-43fdfe-kserve-65f6476d47-wq2jt\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") " pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:42:51.670905 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.670841    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-tmp-dir\") pod \"autores-model-43fdfe-kserve-65f6476d47-wq2jt\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") " pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:42:51.670905 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.670894    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d710d648-e480-4031-b97e-6a9657adfb12-tls-certs\") pod \"autores-model-43fdfe-kserve-65f6476d47-wq2jt\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") " pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:42:51.671028 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.670912    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmsj5\" (UniqueName: \"kubernetes.io/projected/d710d648-e480-4031-b97e-6a9657adfb12-kube-api-access-rmsj5\") pod \"autores-model-43fdfe-kserve-65f6476d47-wq2jt\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") " pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:42:51.671028 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.670970    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-home\") pod \"autores-model-43fdfe-kserve-65f6476d47-wq2jt\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") " pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:42:51.772364 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.772336    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-model-cache\") pod \"autores-model-43fdfe-kserve-65f6476d47-wq2jt\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") " pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:42:51.772495 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.772384    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-dshm\") pod \"autores-model-43fdfe-kserve-65f6476d47-wq2jt\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") " pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:42:51.772495 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.772441    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-tmp-dir\") pod \"autores-model-43fdfe-kserve-65f6476d47-wq2jt\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") " pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:42:51.772581 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.772498    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d710d648-e480-4031-b97e-6a9657adfb12-tls-certs\") pod \"autores-model-43fdfe-kserve-65f6476d47-wq2jt\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") " pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:42:51.772581 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.772522    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmsj5\" (UniqueName: \"kubernetes.io/projected/d710d648-e480-4031-b97e-6a9657adfb12-kube-api-access-rmsj5\") pod \"autores-model-43fdfe-kserve-65f6476d47-wq2jt\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") " pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:42:51.772671 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.772609    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-home\") pod \"autores-model-43fdfe-kserve-65f6476d47-wq2jt\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") " pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:42:51.772821 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.772795    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-model-cache\") pod \"autores-model-43fdfe-kserve-65f6476d47-wq2jt\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") " pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:42:51.772934 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.772881    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-tmp-dir\") pod \"autores-model-43fdfe-kserve-65f6476d47-wq2jt\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") " pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:42:51.772985 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.772969    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-home\") pod \"autores-model-43fdfe-kserve-65f6476d47-wq2jt\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") " pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:42:51.774659 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.774641    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-dshm\") pod \"autores-model-43fdfe-kserve-65f6476d47-wq2jt\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") " pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:42:51.774918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.774901    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d710d648-e480-4031-b97e-6a9657adfb12-tls-certs\") pod \"autores-model-43fdfe-kserve-65f6476d47-wq2jt\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") " pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:42:51.780883 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.780857    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmsj5\" (UniqueName: \"kubernetes.io/projected/d710d648-e480-4031-b97e-6a9657adfb12-kube-api-access-rmsj5\") pod \"autores-model-43fdfe-kserve-65f6476d47-wq2jt\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") " pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:42:51.875433 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.875358    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:42:51.890990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:51.890963    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:42:52.162247 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:52.162134    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"]
Sep 18 06:42:52.164552 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:42:52.164524    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd710d648_e480_4031_b97e_6a9657adfb12.slice/crio-182838cbfad3640fddb035ddf2fae194412fabfea0bedede01a917869c3bfef7 WatchSource:0}: Error finding container 182838cbfad3640fddb035ddf2fae194412fabfea0bedede01a917869c3bfef7: Status 404 returned error can't find the container with id 182838cbfad3640fddb035ddf2fae194412fabfea0bedede01a917869c3bfef7
Sep 18 06:42:53.007011 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:53.006974    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt" event={"ID":"d710d648-e480-4031-b97e-6a9657adfb12","Type":"ContainerStarted","Data":"fcfa0b9d072d93ff3efa535c029401b09598167594f5c70f99536842413ac0be"}
Sep 18 06:42:53.007410 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:53.007029    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt" event={"ID":"d710d648-e480-4031-b97e-6a9657adfb12","Type":"ContainerStarted","Data":"182838cbfad3640fddb035ddf2fae194412fabfea0bedede01a917869c3bfef7"}
Sep 18 06:42:53.023543 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:53.023498    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt" podStartSLOduration=2.02348362 podStartE2EDuration="2.02348362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:42:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:42:53.021675303 +0000 UTC m=+1238.703203048" watchObservedRunningTime="2026-09-18 06:42:53.02348362 +0000 UTC m=+1238.705011363"
Sep 18 06:42:57.484665 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:42:57.484576    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:43:01.891334 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:01.891291    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:43:01.891334 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:01.891340    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:43:01.904021 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:01.903997    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:43:02.066787 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:02.066759    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:43:02.797849 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:02.797812    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:03.541252 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:03.541202    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:04.267131 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:04.267103    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:04.893602 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:04.893565    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:05.112454 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:05.112423    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:05.663702 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:05.663667    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:06.312286 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:06.312253    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:07.128774 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:07.128745    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:07.758349 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:07.758316    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:08.373236 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:08.373183    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:09.227843 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:09.227811    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:09.919195 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:09.919158    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:10.633492 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:10.633439    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:11.262947 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:11.262910    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:17.210951 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.210911    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"]
Sep 18 06:43:17.215744 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.215726    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:17.218415 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.218388    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-654463\"/\"premium-w4-654463-kserve-self-signed-certs\""
Sep 18 06:43:17.224245 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.224205    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"]
Sep 18 06:43:17.234823 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.234793    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:17.339548 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.339507    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v4gs\" (UniqueName: \"kubernetes.io/projected/287848f9-706a-4771-b889-99e2e1247bb8-kube-api-access-6v4gs\") pod \"premium-w4-654463-kserve-6cbc644f8b-vj6xw\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") " pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:17.339732 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.339559    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-tmp-dir\") pod \"premium-w4-654463-kserve-6cbc644f8b-vj6xw\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") " pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:17.339732 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.339578    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-model-cache\") pod \"premium-w4-654463-kserve-6cbc644f8b-vj6xw\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") " pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:17.339732 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.339713    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-home\") pod \"premium-w4-654463-kserve-6cbc644f8b-vj6xw\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") " pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:17.339869 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.339757    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-dshm\") pod \"premium-w4-654463-kserve-6cbc644f8b-vj6xw\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") " pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:17.339869 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.339857    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/287848f9-706a-4771-b889-99e2e1247bb8-tls-certs\") pod \"premium-w4-654463-kserve-6cbc644f8b-vj6xw\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") " pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:17.440933 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.440903    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-tmp-dir\") pod \"premium-w4-654463-kserve-6cbc644f8b-vj6xw\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") " pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:17.441143 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.440947    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-model-cache\") pod \"premium-w4-654463-kserve-6cbc644f8b-vj6xw\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") " pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:17.441143 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.441024    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-home\") pod \"premium-w4-654463-kserve-6cbc644f8b-vj6xw\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") " pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:17.441143 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.441053    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-dshm\") pod \"premium-w4-654463-kserve-6cbc644f8b-vj6xw\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") " pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:17.441143 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.441135    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/287848f9-706a-4771-b889-99e2e1247bb8-tls-certs\") pod \"premium-w4-654463-kserve-6cbc644f8b-vj6xw\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") " pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:17.441370 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.441206    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v4gs\" (UniqueName: \"kubernetes.io/projected/287848f9-706a-4771-b889-99e2e1247bb8-kube-api-access-6v4gs\") pod \"premium-w4-654463-kserve-6cbc644f8b-vj6xw\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") " pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:17.441370 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.441355    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-tmp-dir\") pod \"premium-w4-654463-kserve-6cbc644f8b-vj6xw\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") " pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:17.441441 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.441375    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-home\") pod \"premium-w4-654463-kserve-6cbc644f8b-vj6xw\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") " pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:17.441511 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.441487    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-model-cache\") pod \"premium-w4-654463-kserve-6cbc644f8b-vj6xw\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") " pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:17.443531 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.443508    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-dshm\") pod \"premium-w4-654463-kserve-6cbc644f8b-vj6xw\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") " pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:17.443831 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.443811    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/287848f9-706a-4771-b889-99e2e1247bb8-tls-certs\") pod \"premium-w4-654463-kserve-6cbc644f8b-vj6xw\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") " pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:17.448590 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.448557    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v4gs\" (UniqueName: \"kubernetes.io/projected/287848f9-706a-4771-b889-99e2e1247bb8-kube-api-access-6v4gs\") pod \"premium-w4-654463-kserve-6cbc644f8b-vj6xw\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") " pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:17.525421 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.525347    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:17.802818 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:17.802792    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"]
Sep 18 06:43:17.804401 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:43:17.804371    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod287848f9_706a_4771_b889_99e2e1247bb8.slice/crio-7ffeefbeffa9cf395324362c39cf1cdacc56690520d3c9f115016c19a65008a9 WatchSource:0}: Error finding container 7ffeefbeffa9cf395324362c39cf1cdacc56690520d3c9f115016c19a65008a9: Status 404 returned error can't find the container with id 7ffeefbeffa9cf395324362c39cf1cdacc56690520d3c9f115016c19a65008a9
Sep 18 06:43:18.000962 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:18.000932    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:18.137860 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:18.137780    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw" event={"ID":"287848f9-706a-4771-b889-99e2e1247bb8","Type":"ContainerStarted","Data":"766ead08a0015fe7dd06ae09b7cabced4edaab384dc7b6e501bfe0c94c6a8e9d"}
Sep 18 06:43:18.137860 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:18.137822    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw" event={"ID":"287848f9-706a-4771-b889-99e2e1247bb8","Type":"ContainerStarted","Data":"7ffeefbeffa9cf395324362c39cf1cdacc56690520d3c9f115016c19a65008a9"}
Sep 18 06:43:18.156103 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:18.156053    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw" podStartSLOduration=1.156037596 podStartE2EDuration="1.156037596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:43:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:43:18.154641626 +0000 UTC m=+1263.836169369" watchObservedRunningTime="2026-09-18 06:43:18.156037596 +0000 UTC m=+1263.837565339"
Sep 18 06:43:18.633911 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:18.633879    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:18.702436 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:18.702395    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:18.761115 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:18.761082    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:18.816536 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:18.816506    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:18.954385 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:18.954347    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:19.159311 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:19.159281    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:19.593176 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:19.593147    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:19.692911 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:19.692882    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:19.793154 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:19.793122    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:20.024987 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:20.024953    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:20.661723 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:20.661694    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:21.078682 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:21.078593    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:21.398670 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:21.398642    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:22.107664 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:22.107628    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:22.722886 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:22.722854    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:24.042058 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:24.042020    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:24.845909 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:24.845870    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:25.287591 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:25.287555    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:43:25.658404 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:25.658363    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:26.358261 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:26.358229    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:27.237023 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:27.236988    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:27.422701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:27.422672    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:27.526141 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:27.526065    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:27.526141 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:27.526107    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:27.527870 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:27.527839    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:27.544055 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:27.544020    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:27.622946 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:27.622914    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:27.728588 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:27.728549    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:28.026717 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:28.026684    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:28.126941 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:28.126908    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:28.201275 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:28.201249    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:43:28.827642 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:28.827604    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:31.688302 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:31.688262    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"]
Sep 18 06:43:31.709936 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:31.709907    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:34.342306 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:34.342274    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:34.429792 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:34.429757    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:34.511883 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:34.511850    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:34.563831 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:34.563798    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:34.832825 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:34.832793    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:34.936415 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:34.936378    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:35.622366 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:35.622332    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:36.340907 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:36.340870    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:36.958199 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:36.958161    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:37.573545 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:37.573509    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:39.911357 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:39.911324    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:46.707585 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:46.707534    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt" podUID="d710d648-e480-4031-b97e-6a9657adfb12" containerName="main" containerID="cri-o://fcfa0b9d072d93ff3efa535c029401b09598167594f5c70f99536842413ac0be" gracePeriod=45
Sep 18 06:43:46.959281 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:46.959210    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:43:47.053168 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.053132    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-dshm\") pod \"d710d648-e480-4031-b97e-6a9657adfb12\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") "
Sep 18 06:43:47.053358 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.053176    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-tmp-dir\") pod \"d710d648-e480-4031-b97e-6a9657adfb12\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") "
Sep 18 06:43:47.053358 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.053244    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-home\") pod \"d710d648-e480-4031-b97e-6a9657adfb12\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") "
Sep 18 06:43:47.053358 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.053287    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-model-cache\") pod \"d710d648-e480-4031-b97e-6a9657adfb12\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") "
Sep 18 06:43:47.053358 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.053315    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmsj5\" (UniqueName: \"kubernetes.io/projected/d710d648-e480-4031-b97e-6a9657adfb12-kube-api-access-rmsj5\") pod \"d710d648-e480-4031-b97e-6a9657adfb12\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") "
Sep 18 06:43:47.053583 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.053433    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d710d648-e480-4031-b97e-6a9657adfb12-tls-certs\") pod \"d710d648-e480-4031-b97e-6a9657adfb12\" (UID: \"d710d648-e480-4031-b97e-6a9657adfb12\") "
Sep 18 06:43:47.053583 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.053485    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d710d648-e480-4031-b97e-6a9657adfb12" (UID: "d710d648-e480-4031-b97e-6a9657adfb12"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:43:47.053583 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.053518    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-home" (OuterVolumeSpecName: "home") pod "d710d648-e480-4031-b97e-6a9657adfb12" (UID: "d710d648-e480-4031-b97e-6a9657adfb12"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:43:47.053718 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.053589    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-model-cache" (OuterVolumeSpecName: "model-cache") pod "d710d648-e480-4031-b97e-6a9657adfb12" (UID: "d710d648-e480-4031-b97e-6a9657adfb12"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:43:47.053828 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.053805    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:43:47.053954 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.053830    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:43:47.053954 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.053869    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:43:47.055515 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.055487    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-dshm" (OuterVolumeSpecName: "dshm") pod "d710d648-e480-4031-b97e-6a9657adfb12" (UID: "d710d648-e480-4031-b97e-6a9657adfb12"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:43:47.055623 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.055592    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d710d648-e480-4031-b97e-6a9657adfb12-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d710d648-e480-4031-b97e-6a9657adfb12" (UID: "d710d648-e480-4031-b97e-6a9657adfb12"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:43:47.055679 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.055621    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d710d648-e480-4031-b97e-6a9657adfb12-kube-api-access-rmsj5" (OuterVolumeSpecName: "kube-api-access-rmsj5") pod "d710d648-e480-4031-b97e-6a9657adfb12" (UID: "d710d648-e480-4031-b97e-6a9657adfb12"). InnerVolumeSpecName "kube-api-access-rmsj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:43:47.154633 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.154598    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d710d648-e480-4031-b97e-6a9657adfb12-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:43:47.154633 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.154627    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d710d648-e480-4031-b97e-6a9657adfb12-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:43:47.154821 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.154640    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmsj5\" (UniqueName: \"kubernetes.io/projected/d710d648-e480-4031-b97e-6a9657adfb12-kube-api-access-rmsj5\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:43:47.288973 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.288888    2568 generic.go:356] "Generic (PLEG): container finished" podID="d710d648-e480-4031-b97e-6a9657adfb12" containerID="fcfa0b9d072d93ff3efa535c029401b09598167594f5c70f99536842413ac0be" exitCode=0
Sep 18 06:43:47.288973 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.288941    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"
Sep 18 06:43:47.289145 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.288974    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt" event={"ID":"d710d648-e480-4031-b97e-6a9657adfb12","Type":"ContainerDied","Data":"fcfa0b9d072d93ff3efa535c029401b09598167594f5c70f99536842413ac0be"}
Sep 18 06:43:47.289145 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.289022    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt" event={"ID":"d710d648-e480-4031-b97e-6a9657adfb12","Type":"ContainerDied","Data":"182838cbfad3640fddb035ddf2fae194412fabfea0bedede01a917869c3bfef7"}
Sep 18 06:43:47.289145 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.289042    2568 scope.go:117] "RemoveContainer" containerID="fcfa0b9d072d93ff3efa535c029401b09598167594f5c70f99536842413ac0be"
Sep 18 06:43:47.298438 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.298421    2568 scope.go:117] "RemoveContainer" containerID="fcfa0b9d072d93ff3efa535c029401b09598167594f5c70f99536842413ac0be"
Sep 18 06:43:47.298687 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:43:47.298666    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcfa0b9d072d93ff3efa535c029401b09598167594f5c70f99536842413ac0be\": container with ID starting with fcfa0b9d072d93ff3efa535c029401b09598167594f5c70f99536842413ac0be not found: ID does not exist" containerID="fcfa0b9d072d93ff3efa535c029401b09598167594f5c70f99536842413ac0be"
Sep 18 06:43:47.298754 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.298700    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcfa0b9d072d93ff3efa535c029401b09598167594f5c70f99536842413ac0be"} err="failed to get container status \"fcfa0b9d072d93ff3efa535c029401b09598167594f5c70f99536842413ac0be\": rpc error: code = NotFound desc = could not find container \"fcfa0b9d072d93ff3efa535c029401b09598167594f5c70f99536842413ac0be\": container with ID starting with fcfa0b9d072d93ff3efa535c029401b09598167594f5c70f99536842413ac0be not found: ID does not exist"
Sep 18 06:43:47.308658 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.308634    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"]
Sep 18 06:43:47.312245 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:47.312209    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-43fdfe/autores-model-43fdfe-kserve-65f6476d47-wq2jt"]
Sep 18 06:43:48.544480 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:48.544449    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d710d648-e480-4031-b97e-6a9657adfb12" path="/var/lib/kubelet/pods/d710d648-e480-4031-b97e-6a9657adfb12/volumes"
Sep 18 06:43:51.388650 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:51.388617    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:43:54.042010 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:54.041979    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:59.638527 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:59.638494    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:59.739067 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:59.739033    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:43:59.836533 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:43:59.836501    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:04.982273 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:04.982186    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:44:15.645392 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:15.645354    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:16.240281 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:16.240247    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:16.640390 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:16.640358    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:16.851743 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:16.851662    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"]
Sep 18 06:44:16.852172 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:16.851950    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts" podUID="0f6c5262-48e2-42b1-9e20-45fe353bc292" containerName="istio-proxy" containerID="cri-o://cda555119626fa286662a4162b25e4a6a3198564165791f3eeca9fd6afa48dfa" gracePeriod=30
Sep 18 06:44:16.865629 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:16.865606    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:17.277067 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:17.277031    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:17.552585 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:17.552498    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:17.939055 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:17.939021    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:20.118525 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:20.118490    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:20.751725 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:20.751691    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:20.840996 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:20.840965    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:21.544447 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:21.544403    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:22.074760 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.074727    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:22.111982 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.111920    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:44:22.136282 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.136247    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:22.218074 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.218045    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-podinfo\") pod \"0f6c5262-48e2-42b1-9e20-45fe353bc292\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") "
Sep 18 06:44:22.218299 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.218122    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f6c5262-48e2-42b1-9e20-45fe353bc292-istiod-ca-cert\") pod \"0f6c5262-48e2-42b1-9e20-45fe353bc292\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") "
Sep 18 06:44:22.218299 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.218170    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-data\") pod \"0f6c5262-48e2-42b1-9e20-45fe353bc292\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") "
Sep 18 06:44:22.218431 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.218311    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-workload-certs\") pod \"0f6c5262-48e2-42b1-9e20-45fe353bc292\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") "
Sep 18 06:44:22.218431 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.218375    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-token\") pod \"0f6c5262-48e2-42b1-9e20-45fe353bc292\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") "
Sep 18 06:44:22.218431 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.218409    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-envoy\") pod \"0f6c5262-48e2-42b1-9e20-45fe353bc292\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") "
Sep 18 06:44:22.218599 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.218435    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78c22\" (UniqueName: \"kubernetes.io/projected/0f6c5262-48e2-42b1-9e20-45fe353bc292-kube-api-access-78c22\") pod \"0f6c5262-48e2-42b1-9e20-45fe353bc292\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") "
Sep 18 06:44:22.218599 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.218444    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-data" (OuterVolumeSpecName: "istio-data") pod "0f6c5262-48e2-42b1-9e20-45fe353bc292" (UID: "0f6c5262-48e2-42b1-9e20-45fe353bc292"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:44:22.218599 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.218484    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-workload-socket\") pod \"0f6c5262-48e2-42b1-9e20-45fe353bc292\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") "
Sep 18 06:44:22.218599 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.218514    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-credential-socket\") pod \"0f6c5262-48e2-42b1-9e20-45fe353bc292\" (UID: \"0f6c5262-48e2-42b1-9e20-45fe353bc292\") "
Sep 18 06:44:22.218599 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.218545    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f6c5262-48e2-42b1-9e20-45fe353bc292-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0f6c5262-48e2-42b1-9e20-45fe353bc292" (UID: "0f6c5262-48e2-42b1-9e20-45fe353bc292"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 06:44:22.218599 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.218552    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0f6c5262-48e2-42b1-9e20-45fe353bc292" (UID: "0f6c5262-48e2-42b1-9e20-45fe353bc292"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:44:22.218880 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.218716    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0f6c5262-48e2-42b1-9e20-45fe353bc292" (UID: "0f6c5262-48e2-42b1-9e20-45fe353bc292"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:44:22.218880 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.218773    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0f6c5262-48e2-42b1-9e20-45fe353bc292" (UID: "0f6c5262-48e2-42b1-9e20-45fe353bc292"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:44:22.218971 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.218919    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0f6c5262-48e2-42b1-9e20-45fe353bc292-istiod-ca-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:44:22.218971 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.218930    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-data\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:44:22.218971 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.218938    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-workload-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:44:22.218971 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.218947    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-workload-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:44:22.218971 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.218955    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-credential-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:44:22.220576 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.220502    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:22.220843 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.220815    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0f6c5262-48e2-42b1-9e20-45fe353bc292" (UID: "0f6c5262-48e2-42b1-9e20-45fe353bc292"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 06:44:22.220943 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.220851    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f6c5262-48e2-42b1-9e20-45fe353bc292-kube-api-access-78c22" (OuterVolumeSpecName: "kube-api-access-78c22") pod "0f6c5262-48e2-42b1-9e20-45fe353bc292" (UID: "0f6c5262-48e2-42b1-9e20-45fe353bc292"). InnerVolumeSpecName "kube-api-access-78c22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:44:22.220943 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.220895    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-token" (OuterVolumeSpecName: "istio-token") pod "0f6c5262-48e2-42b1-9e20-45fe353bc292" (UID: "0f6c5262-48e2-42b1-9e20-45fe353bc292"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:44:22.221055 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.221035    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0f6c5262-48e2-42b1-9e20-45fe353bc292" (UID: "0f6c5262-48e2-42b1-9e20-45fe353bc292"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:44:22.320122 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.320084    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-podinfo\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:44:22.320122 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.320120    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-token\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:44:22.320427 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.320134    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0f6c5262-48e2-42b1-9e20-45fe353bc292-istio-envoy\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:44:22.320427 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.320147    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78c22\" (UniqueName: \"kubernetes.io/projected/0f6c5262-48e2-42b1-9e20-45fe353bc292-kube-api-access-78c22\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:44:22.320557 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.320529    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:22.477055 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.477024    2568 generic.go:356] "Generic (PLEG): container finished" podID="0f6c5262-48e2-42b1-9e20-45fe353bc292" containerID="cda555119626fa286662a4162b25e4a6a3198564165791f3eeca9fd6afa48dfa" exitCode=0
Sep 18 06:44:22.477245 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.477076    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"
Sep 18 06:44:22.477245 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.477108    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts" event={"ID":"0f6c5262-48e2-42b1-9e20-45fe353bc292","Type":"ContainerDied","Data":"cda555119626fa286662a4162b25e4a6a3198564165791f3eeca9fd6afa48dfa"}
Sep 18 06:44:22.477245 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.477171    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts" event={"ID":"0f6c5262-48e2-42b1-9e20-45fe353bc292","Type":"ContainerDied","Data":"d004cfde3a257241456b187fbb8b5870f0e360074bd12f54414aada33c82c51e"}
Sep 18 06:44:22.477245 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.477195    2568 scope.go:117] "RemoveContainer" containerID="cda555119626fa286662a4162b25e4a6a3198564165791f3eeca9fd6afa48dfa"
Sep 18 06:44:22.486835 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.486818    2568 scope.go:117] "RemoveContainer" containerID="3e8a27f6d405745b8130f7951277864691983c05e151a19a4c0eb8f439eb7f5c"
Sep 18 06:44:22.494572 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.494557    2568 scope.go:117] "RemoveContainer" containerID="cda555119626fa286662a4162b25e4a6a3198564165791f3eeca9fd6afa48dfa"
Sep 18 06:44:22.494825 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:44:22.494807    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cda555119626fa286662a4162b25e4a6a3198564165791f3eeca9fd6afa48dfa\": container with ID starting with cda555119626fa286662a4162b25e4a6a3198564165791f3eeca9fd6afa48dfa not found: ID does not exist" containerID="cda555119626fa286662a4162b25e4a6a3198564165791f3eeca9fd6afa48dfa"
Sep 18 06:44:22.494866 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.494833    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cda555119626fa286662a4162b25e4a6a3198564165791f3eeca9fd6afa48dfa"} err="failed to get container status \"cda555119626fa286662a4162b25e4a6a3198564165791f3eeca9fd6afa48dfa\": rpc error: code = NotFound desc = could not find container \"cda555119626fa286662a4162b25e4a6a3198564165791f3eeca9fd6afa48dfa\": container with ID starting with cda555119626fa286662a4162b25e4a6a3198564165791f3eeca9fd6afa48dfa not found: ID does not exist"
Sep 18 06:44:22.494866 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.494848    2568 scope.go:117] "RemoveContainer" containerID="3e8a27f6d405745b8130f7951277864691983c05e151a19a4c0eb8f439eb7f5c"
Sep 18 06:44:22.495089 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:44:22.495074    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e8a27f6d405745b8130f7951277864691983c05e151a19a4c0eb8f439eb7f5c\": container with ID starting with 3e8a27f6d405745b8130f7951277864691983c05e151a19a4c0eb8f439eb7f5c not found: ID does not exist" containerID="3e8a27f6d405745b8130f7951277864691983c05e151a19a4c0eb8f439eb7f5c"
Sep 18 06:44:22.495148 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.495093    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e8a27f6d405745b8130f7951277864691983c05e151a19a4c0eb8f439eb7f5c"} err="failed to get container status \"3e8a27f6d405745b8130f7951277864691983c05e151a19a4c0eb8f439eb7f5c\": rpc error: code = NotFound desc = could not find container \"3e8a27f6d405745b8130f7951277864691983c05e151a19a4c0eb8f439eb7f5c\": container with ID starting with 3e8a27f6d405745b8130f7951277864691983c05e151a19a4c0eb8f439eb7f5c not found: ID does not exist"
Sep 18 06:44:22.509055 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.509025    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"]
Sep 18 06:44:22.522605 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.522582    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts"]
Sep 18 06:44:22.543462 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:22.543438    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f6c5262-48e2-42b1-9e20-45fe353bc292" path="/var/lib/kubelet/pods/0f6c5262-48e2-42b1-9e20-45fe353bc292/volumes"
Sep 18 06:44:23.138704 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:23.138673    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:23.691544 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:23.691511    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:44:23.775568 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:23.775536    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:23.876483 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:23.876449    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:23.975972 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:23.975895    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:24.231764 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:24.231672    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:24.335874 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:24.335836    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:32.735663 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:32.735583    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:33.470156 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:33.470113    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:39.235051 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:39.234977    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:39.293693 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:39.293664    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:39.374791 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:39.374755    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:39.470357 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:39.470313    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:39.570444 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:39.570369    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:40.138993 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:40.138958    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:40.236121 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:40.236088    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:40.336106 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:40.336071    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:41.368502 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.368470    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"]
Sep 18 06:44:41.369022 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.368995    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d710d648-e480-4031-b97e-6a9657adfb12" containerName="main"
Sep 18 06:44:41.369022 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.369017    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d710d648-e480-4031-b97e-6a9657adfb12" containerName="main"
Sep 18 06:44:41.369250 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.369043    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f6c5262-48e2-42b1-9e20-45fe353bc292" containerName="istio-proxy"
Sep 18 06:44:41.369250 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.369052    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f6c5262-48e2-42b1-9e20-45fe353bc292" containerName="istio-proxy"
Sep 18 06:44:41.369250 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.369189    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f6c5262-48e2-42b1-9e20-45fe353bc292" containerName="istio-proxy"
Sep 18 06:44:41.369250 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.369204    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d710d648-e480-4031-b97e-6a9657adfb12" containerName="main"
Sep 18 06:44:41.371763 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.371745    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:41.375140 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.375116    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-708b6f\"/\"openshift-service-ca.crt\""
Sep 18 06:44:41.375140 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.375126    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-708b6f\"/\"test-model-708b6f-kserve-self-signed-certs\""
Sep 18 06:44:41.375333 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.375141    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-708b6f\"/\"kube-root-ca.crt\""
Sep 18 06:44:41.375333 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.375149    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-708b6f\"/\"default-dockercfg-qfdzn\""
Sep 18 06:44:41.381910 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.381889    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"]
Sep 18 06:44:41.520828 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.520789    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ae10764-63e1-4de0-9961-ccb4c458e4d9-tls-certs\") pod \"test-model-708b6f-kserve-67746c4b67-27n2n\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") " pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:41.520828 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.520830    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-home\") pod \"test-model-708b6f-kserve-67746c4b67-27n2n\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") " pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:41.521097 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.520857    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-dshm\") pod \"test-model-708b6f-kserve-67746c4b67-27n2n\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") " pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:41.521097 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.520998    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-tmp-dir\") pod \"test-model-708b6f-kserve-67746c4b67-27n2n\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") " pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:41.521097 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.521074    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-model-cache\") pod \"test-model-708b6f-kserve-67746c4b67-27n2n\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") " pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:41.521272 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.521109    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wj2z\" (UniqueName: \"kubernetes.io/projected/7ae10764-63e1-4de0-9961-ccb4c458e4d9-kube-api-access-7wj2z\") pod \"test-model-708b6f-kserve-67746c4b67-27n2n\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") " pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:41.577606 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.577577    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:41.622315 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.622241    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-tmp-dir\") pod \"test-model-708b6f-kserve-67746c4b67-27n2n\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") " pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:41.622315 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.622307    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-model-cache\") pod \"test-model-708b6f-kserve-67746c4b67-27n2n\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") " pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:41.622487 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.622337    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wj2z\" (UniqueName: \"kubernetes.io/projected/7ae10764-63e1-4de0-9961-ccb4c458e4d9-kube-api-access-7wj2z\") pod \"test-model-708b6f-kserve-67746c4b67-27n2n\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") " pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:41.622487 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.622388    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ae10764-63e1-4de0-9961-ccb4c458e4d9-tls-certs\") pod \"test-model-708b6f-kserve-67746c4b67-27n2n\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") " pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:41.622487 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.622417    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-home\") pod \"test-model-708b6f-kserve-67746c4b67-27n2n\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") " pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:41.622487 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.622443    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-dshm\") pod \"test-model-708b6f-kserve-67746c4b67-27n2n\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") " pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:41.622710 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.622554    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-tmp-dir\") pod \"test-model-708b6f-kserve-67746c4b67-27n2n\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") " pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:41.622710 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.622646    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-model-cache\") pod \"test-model-708b6f-kserve-67746c4b67-27n2n\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") " pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:41.622823 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.622756    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-home\") pod \"test-model-708b6f-kserve-67746c4b67-27n2n\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") " pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:41.624644 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.624626    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-dshm\") pod \"test-model-708b6f-kserve-67746c4b67-27n2n\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") " pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:41.624870 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.624853    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ae10764-63e1-4de0-9961-ccb4c458e4d9-tls-certs\") pod \"test-model-708b6f-kserve-67746c4b67-27n2n\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") " pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:41.629348 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.629322    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wj2z\" (UniqueName: \"kubernetes.io/projected/7ae10764-63e1-4de0-9961-ccb4c458e4d9-kube-api-access-7wj2z\") pod \"test-model-708b6f-kserve-67746c4b67-27n2n\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") " pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:41.680876 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.680846    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:41.727015 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.726978    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:41.966373 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:41.965022    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"]
Sep 18 06:44:41.970139 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:44:41.970111    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ae10764_63e1_4de0_9961_ccb4c458e4d9.slice/crio-d3fe0386c9993e79d1361b95d29f7b27927fb50cb74708952e9273b6f0693bce WatchSource:0}: Error finding container d3fe0386c9993e79d1361b95d29f7b27927fb50cb74708952e9273b6f0693bce: Status 404 returned error can't find the container with id d3fe0386c9993e79d1361b95d29f7b27927fb50cb74708952e9273b6f0693bce
Sep 18 06:44:42.585968 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:42.585933    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n" event={"ID":"7ae10764-63e1-4de0-9961-ccb4c458e4d9","Type":"ContainerStarted","Data":"ef88beba0afb920924574d5ecb21148ada76f6f6c152f4dbdff6f1e1b0846bdf"}
Sep 18 06:44:42.586457 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:42.585980    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n" event={"ID":"7ae10764-63e1-4de0-9961-ccb4c458e4d9","Type":"ContainerStarted","Data":"d3fe0386c9993e79d1361b95d29f7b27927fb50cb74708952e9273b6f0693bce"}
Sep 18 06:44:42.600676 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:42.600626    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n" podStartSLOduration=1.600608436 podStartE2EDuration="1.600608436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:44:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:44:42.600412908 +0000 UTC m=+1348.281940648" watchObservedRunningTime="2026-09-18 06:44:42.600608436 +0000 UTC m=+1348.282136180"
Sep 18 06:44:46.129652 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.129615    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:46.483836 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.483800    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"]
Sep 18 06:44:46.484488 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.484469    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f6c5262-48e2-42b1-9e20-45fe353bc292" containerName="istio-proxy"
Sep 18 06:44:46.484868 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.484492    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f6c5262-48e2-42b1-9e20-45fe353bc292" containerName="istio-proxy"
Sep 18 06:44:46.484868 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.484664    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f6c5262-48e2-42b1-9e20-45fe353bc292" containerName="istio-proxy"
Sep 18 06:44:46.488458 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.488432    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:46.490858 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.490833    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-654463\"/\"e2e-conflict-a-24f27817-kserve-self-signed-certs\""
Sep 18 06:44:46.496150 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.496114    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"]
Sep 18 06:44:46.521408 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.521379    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:46.576055 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.576023    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-tmp-dir\") pod \"e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") " pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:46.576257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.576065    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-model-cache\") pod \"e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") " pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:46.576257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.576087    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-home\") pod \"e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") " pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:46.576257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.576148    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0059a78e-a837-4c30-a219-cdffed4da546-tls-certs\") pod \"e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") " pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:46.576257 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.576232    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jtgg\" (UniqueName: \"kubernetes.io/projected/0059a78e-a837-4c30-a219-cdffed4da546-kube-api-access-6jtgg\") pod \"e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") " pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:46.576473 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.576296    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-dshm\") pod \"e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") " pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:46.677567 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.677531    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-tmp-dir\") pod \"e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") " pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:46.677745 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.677591    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-model-cache\") pod \"e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") " pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:46.677745 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.677618    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-home\") pod \"e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") " pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:46.677745 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.677644    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0059a78e-a837-4c30-a219-cdffed4da546-tls-certs\") pod \"e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") " pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:46.677745 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.677677    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jtgg\" (UniqueName: \"kubernetes.io/projected/0059a78e-a837-4c30-a219-cdffed4da546-kube-api-access-6jtgg\") pod \"e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") " pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:46.677745 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.677737    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-dshm\") pod \"e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") " pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:46.678063 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.678037    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-tmp-dir\") pod \"e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") " pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:46.678063 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.678057    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-home\") pod \"e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") " pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:46.678173 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.678111    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-model-cache\") pod \"e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") " pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:46.679953 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.679930    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-dshm\") pod \"e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") " pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:46.680496 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.680473    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0059a78e-a837-4c30-a219-cdffed4da546-tls-certs\") pod \"e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") " pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:46.685145 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.685122    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jtgg\" (UniqueName: \"kubernetes.io/projected/0059a78e-a837-4c30-a219-cdffed4da546-kube-api-access-6jtgg\") pod \"e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") " pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:46.800471 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.800392    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:46.955120 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:46.955081    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:47.073211 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:47.073184    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"]
Sep 18 06:44:47.074821 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:44:47.074794    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0059a78e_a837_4c30_a219_cdffed4da546.slice/crio-7ed4cca949c755df70d93e66616ead0741b7f07a7bbf6c846394cb7be485a9db WatchSource:0}: Error finding container 7ed4cca949c755df70d93e66616ead0741b7f07a7bbf6c846394cb7be485a9db: Status 404 returned error can't find the container with id 7ed4cca949c755df70d93e66616ead0741b7f07a7bbf6c846394cb7be485a9db
Sep 18 06:44:47.561582 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:47.561552    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:47.615071 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:47.615035    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd" event={"ID":"0059a78e-a837-4c30-a219-cdffed4da546","Type":"ContainerStarted","Data":"9bdcf2907c6dbc4661e0ec993f5fb23b7afb2d206920d97d35196026d2c1e10e"}
Sep 18 06:44:47.615255 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:47.615084    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd" event={"ID":"0059a78e-a837-4c30-a219-cdffed4da546","Type":"ContainerStarted","Data":"7ed4cca949c755df70d93e66616ead0741b7f07a7bbf6c846394cb7be485a9db"}
Sep 18 06:44:47.620848 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:47.620825    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:47.633242 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:47.633179    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd" podStartSLOduration=1.633164094 podStartE2EDuration="1.633164094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:44:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:44:47.631040212 +0000 UTC m=+1353.312567959" watchObservedRunningTime="2026-09-18 06:44:47.633164094 +0000 UTC m=+1353.314691838"
Sep 18 06:44:47.734625 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:47.734572    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:48.195525 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:48.195489    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:48.373451 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:48.373414    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:48.592834 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:48.592751    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:44:49.030484 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:49.030450    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:49.790903 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:49.790856    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:50.737593 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:50.737551    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:51.681310 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:51.681278    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:51.681719 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:51.681318    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:51.693732 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:51.693710    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:52.652905 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:52.652876    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:44:53.813677 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:53.813629    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:53.914820 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:53.914789    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:54.022306 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:54.022275    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:54.379049 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:54.379019    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:54.546405 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:54.546367    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:56.800524 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:56.800497    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:56.800994 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:56.800536    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:56.814550 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:56.814526    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:57.678141 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:57.678116    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:44:58.922573 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:58.922538    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:59.579549 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:59.579506    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:59.678874 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:59.678848    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:44:59.779308 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:44:59.779273    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:00.144123 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:00.144083    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7c4f4b-29828565-f84cn"]
Sep 18 06:45:00.147995 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:00.147972    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7c4f4b-29828565-f84cn"
Sep 18 06:45:00.150461 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:00.150443    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-88t7h\""
Sep 18 06:45:00.160672 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:00.160647    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7c4f4b-29828565-f84cn"
Sep 18 06:45:00.160878 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:00.160846    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7c4f4b-29828565-f84cn"]
Sep 18 06:45:00.167641 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:00.167617    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-946609-29828565-dr9tz"]
Sep 18 06:45:00.171828 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:00.171810    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-946609-29828565-dr9tz"
Sep 18 06:45:00.179673 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:00.179652    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-946609-29828565-dr9tz"]
Sep 18 06:45:00.201744 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:00.201719    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-946609-29828565-dr9tz"
Sep 18 06:45:00.439871 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:00.439845    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7c4f4b-29828565-f84cn"]
Sep 18 06:45:00.441254 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:45:00.441196    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28ce143b_6486_4db0_b395_56c03da9e731.slice/crio-d0b359ac8551e71187ec441097e94c2bfa8052dbf805d47d0fcd2a473a7c5846 WatchSource:0}: Error finding container d0b359ac8551e71187ec441097e94c2bfa8052dbf805d47d0fcd2a473a7c5846: Status 404 returned error can't find the container with id d0b359ac8551e71187ec441097e94c2bfa8052dbf805d47d0fcd2a473a7c5846
Sep 18 06:45:00.443412 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:00.443391    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 06:45:00.479626 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:00.479594    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-946609-29828565-dr9tz"]
Sep 18 06:45:00.481070 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:45:00.481048    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb166992_c8bd_4419_945d_f18bd7726a2f.slice/crio-07df3909552bd80bcb308d642234e5651ff5fc5d1df01fafb0604b95339befe7 WatchSource:0}: Error finding container 07df3909552bd80bcb308d642234e5651ff5fc5d1df01fafb0604b95339befe7: Status 404 returned error can't find the container with id 07df3909552bd80bcb308d642234e5651ff5fc5d1df01fafb0604b95339befe7
Sep 18 06:45:00.557713 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:00.557682    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:00.682560 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:00.682526    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-946609-29828565-dr9tz" event={"ID":"eb166992-c8bd-4419-945d-f18bd7726a2f","Type":"ContainerStarted","Data":"07df3909552bd80bcb308d642234e5651ff5fc5d1df01fafb0604b95339befe7"}
Sep 18 06:45:00.683711 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:00.683684    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7c4f4b-29828565-f84cn" event={"ID":"28ce143b-6486-4db0-b395-56c03da9e731","Type":"ContainerStarted","Data":"d0b359ac8551e71187ec441097e94c2bfa8052dbf805d47d0fcd2a473a7c5846"}
Sep 18 06:45:01.321210 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:01.321174    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:01.942546 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:01.942506    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:02.741342 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:02.741279    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:03.509494 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:03.509403    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:03.609614 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:03.609571    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:03.706648 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:03.706613    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-946609-29828565-dr9tz" event={"ID":"eb166992-c8bd-4419-945d-f18bd7726a2f","Type":"ContainerStarted","Data":"c6dcd496bac232860829fbab928a0d443b3f099abf9212749ea02fe483126564"}
Sep 18 06:45:03.708283 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:03.708251    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7c4f4b-29828565-f84cn" event={"ID":"28ce143b-6486-4db0-b395-56c03da9e731","Type":"ContainerStarted","Data":"dedbd10e8110ab29f8475ce52e30144a6babc53380b8322796542faaaf769f76"}
Sep 18 06:45:03.721759 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:03.721715    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-946609-29828565-dr9tz" podStartSLOduration=1.566972958 podStartE2EDuration="3.721701937s" totalImagesPullingTime="2.154728979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:45:03.720656416 +0000 UTC m=+1369.402184160" watchObservedRunningTime="2026-09-18 06:45:03.721701937 +0000 UTC m=+1369.403229680"
Sep 18 06:45:03.733478 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:03.733428    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7c4f4b-29828565-f84cn" podStartSLOduration=1.538279626 podStartE2EDuration="3.73340929s" totalImagesPullingTime="2.195129664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:45:03.733384962 +0000 UTC m=+1369.414912707" watchObservedRunningTime="2026-09-18 06:45:03.73340929 +0000 UTC m=+1369.414937038"
Sep 18 06:45:03.944363 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:03.944330    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:04.033487 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:04.033425    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:04.306417 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:04.306377    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:04.715163 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:04.715127    2568 generic.go:356] "Generic (PLEG): container finished" podID="eb166992-c8bd-4419-945d-f18bd7726a2f" containerID="c6dcd496bac232860829fbab928a0d443b3f099abf9212749ea02fe483126564" exitCode=0
Sep 18 06:45:04.715390 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:04.715241    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-946609-29828565-dr9tz" event={"ID":"eb166992-c8bd-4419-945d-f18bd7726a2f","Type":"ContainerDied","Data":"c6dcd496bac232860829fbab928a0d443b3f099abf9212749ea02fe483126564"}
Sep 18 06:45:04.716730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:04.716710    2568 generic.go:356] "Generic (PLEG): container finished" podID="28ce143b-6486-4db0-b395-56c03da9e731" containerID="dedbd10e8110ab29f8475ce52e30144a6babc53380b8322796542faaaf769f76" exitCode=0
Sep 18 06:45:04.716835 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:04.716764    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7c4f4b-29828565-f84cn" event={"ID":"28ce143b-6486-4db0-b395-56c03da9e731","Type":"ContainerDied","Data":"dedbd10e8110ab29f8475ce52e30144a6babc53380b8322796542faaaf769f76"}
Sep 18 06:45:05.874603 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:05.874579    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-946609-29828565-dr9tz"
Sep 18 06:45:05.907054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:05.907031    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7c4f4b-29828565-f84cn"
Sep 18 06:45:06.729056 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:06.729032    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7c4f4b-29828565-f84cn"
Sep 18 06:45:06.729262 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:06.729074    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7c4f4b-29828565-f84cn" event={"ID":"28ce143b-6486-4db0-b395-56c03da9e731","Type":"ContainerDied","Data":"d0b359ac8551e71187ec441097e94c2bfa8052dbf805d47d0fcd2a473a7c5846"}
Sep 18 06:45:06.729262 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:06.729123    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0b359ac8551e71187ec441097e94c2bfa8052dbf805d47d0fcd2a473a7c5846"
Sep 18 06:45:06.730734 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:06.730713    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-946609-29828565-dr9tz"
Sep 18 06:45:06.730734 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:06.730717    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-946609-29828565-dr9tz" event={"ID":"eb166992-c8bd-4419-945d-f18bd7726a2f","Type":"ContainerDied","Data":"07df3909552bd80bcb308d642234e5651ff5fc5d1df01fafb0604b95339befe7"}
Sep 18 06:45:06.730929 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:06.730775    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="07df3909552bd80bcb308d642234e5651ff5fc5d1df01fafb0604b95339befe7"
Sep 18 06:45:14.994878 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:14.994845    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:45:25.992790 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:25.992753    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:26.626588 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:26.626537    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:27.345002 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:27.344966    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:27.959726 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:27.959694    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:28.688745 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:28.688696    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:29.429778 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:29.429732    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:29.667713 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:29.667672    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:29.767401 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:29.767320    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:29.869526 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:29.869495    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:30.055738 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:30.055664    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:30.294738 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:30.294707    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:30.391548 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:30.391435    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:30.729776 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:30.729725    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:36.479986 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:36.479956    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:36.580446 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:36.580409    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:37.627135 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:37.627070    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:39.092002 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:39.091941    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:39.191442 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:39.191405    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:39.675850 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:39.675818    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:40.308742 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:40.308707    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:41.127961 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:41.127928    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:41.739787 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:41.739754    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:43.385619 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:43.385587    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:45:47.348718 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:47.348673    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:47.461802 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:47.461765    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:47.563884 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:47.563856    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:47.616425 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:47.616354    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:47.922065 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:47.922027    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:48.023015 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:48.022986    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:48.746965 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:48.746930    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:49.474821 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:49.474772    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:50.230939 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:50.230904    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:50.851727 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:50.851682    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:51.568083 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:51.568041    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:53.661846 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:53.661803    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:53.762502 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:53.762466    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:53.894213 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:53.894178    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:54.705332 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:54.705289    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:55.282951 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:55.282918    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:56.412531 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:56.412471    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:57.034604 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:57.034572    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:45:57.792778 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:45:57.792746    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:00.001294 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:00.001260    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:00.112398 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:00.112366    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:00.202083 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:00.202048    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:00.310828 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:00.310735    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:06.002647 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:06.002614    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:06.099150 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:06.099118    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:06.855372 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:06.855336    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:07.177071 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:07.177030    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:07.767274 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:07.767235    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:09.753756 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:09.753725    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:10.074584 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:10.074496    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:10.423201 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:10.423166    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:10.527233 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:10.527196    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:10.624160 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:10.624132    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:11.997445 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:11.997413    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:12.097152 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:12.097123    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:12.752392 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:12.752362    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:13.398584 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:13.398555    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:13.685794 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:13.685712    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:46:14.134798 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.134710    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"]
Sep 18 06:46:14.135542 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.135507    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb166992-c8bd-4419-945d-f18bd7726a2f" containerName="cleanup"
Sep 18 06:46:14.135542 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.135532    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb166992-c8bd-4419-945d-f18bd7726a2f" containerName="cleanup"
Sep 18 06:46:14.135741 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.135569    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28ce143b-6486-4db0-b395-56c03da9e731" containerName="cleanup"
Sep 18 06:46:14.135741 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.135578    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="28ce143b-6486-4db0-b395-56c03da9e731" containerName="cleanup"
Sep 18 06:46:14.135741 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.135669    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb166992-c8bd-4419-945d-f18bd7726a2f" containerName="cleanup"
Sep 18 06:46:14.135741 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.135685    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="28ce143b-6486-4db0-b395-56c03da9e731" containerName="cleanup"
Sep 18 06:46:14.138791 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.138767    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"
Sep 18 06:46:14.141266 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.141245    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-bac224-serving-cert\""
Sep 18 06:46:14.148592 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.148122    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"]
Sep 18 06:46:14.164604 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.164568    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:14.189526 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.189502    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a819377f-5ecf-4063-9e9d-bbc234fe06b9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr\" (UID: \"a819377f-5ecf-4063-9e9d-bbc234fe06b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"
Sep 18 06:46:14.189655 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.189536    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a819377f-5ecf-4063-9e9d-bbc234fe06b9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr\" (UID: \"a819377f-5ecf-4063-9e9d-bbc234fe06b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"
Sep 18 06:46:14.189727 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.189665    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sshzq\" (UniqueName: \"kubernetes.io/projected/a819377f-5ecf-4063-9e9d-bbc234fe06b9-kube-api-access-sshzq\") pod \"maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr\" (UID: \"a819377f-5ecf-4063-9e9d-bbc234fe06b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"
Sep 18 06:46:14.189727 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.189712    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a819377f-5ecf-4063-9e9d-bbc234fe06b9-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr\" (UID: \"a819377f-5ecf-4063-9e9d-bbc234fe06b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"
Sep 18 06:46:14.225273 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.225240    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:14.290649 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.290615    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sshzq\" (UniqueName: \"kubernetes.io/projected/a819377f-5ecf-4063-9e9d-bbc234fe06b9-kube-api-access-sshzq\") pod \"maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr\" (UID: \"a819377f-5ecf-4063-9e9d-bbc234fe06b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"
Sep 18 06:46:14.290831 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.290659    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a819377f-5ecf-4063-9e9d-bbc234fe06b9-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr\" (UID: \"a819377f-5ecf-4063-9e9d-bbc234fe06b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"
Sep 18 06:46:14.290831 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.290776    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a819377f-5ecf-4063-9e9d-bbc234fe06b9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr\" (UID: \"a819377f-5ecf-4063-9e9d-bbc234fe06b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"
Sep 18 06:46:14.290831 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.290806    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a819377f-5ecf-4063-9e9d-bbc234fe06b9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr\" (UID: \"a819377f-5ecf-4063-9e9d-bbc234fe06b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"
Sep 18 06:46:14.291715 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.291687    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a819377f-5ecf-4063-9e9d-bbc234fe06b9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr\" (UID: \"a819377f-5ecf-4063-9e9d-bbc234fe06b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"
Sep 18 06:46:14.294339 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.294314    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a819377f-5ecf-4063-9e9d-bbc234fe06b9-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr\" (UID: \"a819377f-5ecf-4063-9e9d-bbc234fe06b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"
Sep 18 06:46:14.294500 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.294474    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a819377f-5ecf-4063-9e9d-bbc234fe06b9-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr\" (UID: \"a819377f-5ecf-4063-9e9d-bbc234fe06b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"
Sep 18 06:46:14.300081 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.300050    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sshzq\" (UniqueName: \"kubernetes.io/projected/a819377f-5ecf-4063-9e9d-bbc234fe06b9-kube-api-access-sshzq\") pod \"maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr\" (UID: \"a819377f-5ecf-4063-9e9d-bbc234fe06b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"
Sep 18 06:46:14.325459 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.325435    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:14.453005 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.452972    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"
Sep 18 06:46:14.632043 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.632003    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:14.723988 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.723951    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"]
Sep 18 06:46:14.725367 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:14.725347    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:14.725982 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:46:14.725959    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda819377f_5ecf_4063_9e9d_bbc234fe06b9.slice/crio-86e33b11830bc4ccd78e0703e17546cf5c6aff49e047b78a9a8113c9e701780c WatchSource:0}: Error finding container 86e33b11830bc4ccd78e0703e17546cf5c6aff49e047b78a9a8113c9e701780c: Status 404 returned error can't find the container with id 86e33b11830bc4ccd78e0703e17546cf5c6aff49e047b78a9a8113c9e701780c
Sep 18 06:46:15.118603 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:15.118514    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr" event={"ID":"a819377f-5ecf-4063-9e9d-bbc234fe06b9","Type":"ContainerStarted","Data":"86e33b11830bc4ccd78e0703e17546cf5c6aff49e047b78a9a8113c9e701780c"}
Sep 18 06:46:16.126010 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:16.125973    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr" event={"ID":"a819377f-5ecf-4063-9e9d-bbc234fe06b9","Type":"ContainerStarted","Data":"f772243021110256239a242e0686878b0d7734b33740501034e6e01ebf5af814"}
Sep 18 06:46:16.126544 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:16.126069    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"
Sep 18 06:46:16.143892 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:16.143843    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr" podStartSLOduration=1.685470831 podStartE2EDuration="2.143827851s" totalImagesPullingTime="458.35702ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:46:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:46:16.140495683 +0000 UTC m=+1441.822023431" watchObservedRunningTime="2026-09-18 06:46:16.143827851 +0000 UTC m=+1441.825355657"
Sep 18 06:46:19.228850 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:19.228820    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:22.138026 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:22.137997    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"
Sep 18 06:46:29.612397 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:29.612364    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:33.207904 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:33.207872    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:33.437289 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:33.437255    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:34.228891 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:34.228857    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:34.917105 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:34.917072    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:35.551618 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:35.551583    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:36.218778 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:36.218742    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:36.926982 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:36.926949    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:37.542954 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:37.542924    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:38.233154 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:38.233102    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:38.847280 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:38.847239    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:39.619161 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:39.619134    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:40.249599 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:40.249567    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:40.510646 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:40.510558    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:40.875211 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:40.875120    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:40.968783 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:40.968756    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:41.128587 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.128495    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"]
Sep 18 06:46:41.132911 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.132880    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.135462 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.135441    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-87d618-openshift-default-dockercfg-rbbrd\""
Sep 18 06:46:41.142738 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.142718    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"]
Sep 18 06:46:41.145355 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.145331    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:41.164121 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.164100    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istiod-ca-cert\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.164231 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.164132    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-data\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.164231 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.164176    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-podinfo\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.164231 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.164199    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-token\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.164420 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.164261    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-envoy\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.164420 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.164278    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-workload-certs\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.164420 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.164300    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-workload-socket\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.164420 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.164332    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk9f2\" (UniqueName: \"kubernetes.io/projected/52f56e47-435a-44dc-be93-b6cd22b2ce2e-kube-api-access-zk9f2\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.164420 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.164367    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-credential-socket\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.269274 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.269240    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-credential-socket\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.269461 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.269319    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istiod-ca-cert\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.269461 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.269355    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-data\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.269461 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.269426    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-podinfo\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.269640 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.269462    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-token\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.269640 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.269504    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-envoy\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.269640 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.269527    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-workload-certs\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.269640 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.269558    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-workload-socket\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.269640 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.269607    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk9f2\" (UniqueName: \"kubernetes.io/projected/52f56e47-435a-44dc-be93-b6cd22b2ce2e-kube-api-access-zk9f2\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.269908 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.269713    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-credential-socket\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.269908 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.269792    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-data\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.270094 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.270068    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istiod-ca-cert\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.270371 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.270184    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-workload-certs\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.270371 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.270305    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-workload-socket\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.272044 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.272022    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-envoy\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.272153 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.272132    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-podinfo\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.276980 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.276956    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-token\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.277372 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.277351    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk9f2\" (UniqueName: \"kubernetes.io/projected/52f56e47-435a-44dc-be93-b6cd22b2ce2e-kube-api-access-zk9f2\") pod \"e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") " pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.431609 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.431570    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:41.445570 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.445544    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:41.727497 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.727472    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"]
Sep 18 06:46:41.728700 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:46:41.728673    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52f56e47_435a_44dc_be93_b6cd22b2ce2e.slice/crio-29c2c2899b32a319a368c17c4486be6dbd6455f9a87ddd2f5dd7918ea9674828 WatchSource:0}: Error finding container 29c2c2899b32a319a368c17c4486be6dbd6455f9a87ddd2f5dd7918ea9674828: Status 404 returned error can't find the container with id 29c2c2899b32a319a368c17c4486be6dbd6455f9a87ddd2f5dd7918ea9674828
Sep 18 06:46:41.731428 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.731386    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:46:41.731536 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.731473    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:46:41.731601 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.731536    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:46:41.928279 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:41.928247    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:42.281769 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:42.281700    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn" event={"ID":"52f56e47-435a-44dc-be93-b6cd22b2ce2e","Type":"ContainerStarted","Data":"fb43b75821a5e34c7308716646637cf81fae806601a9489e52cf2c65f92088aa"}
Sep 18 06:46:42.281925 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:42.281782    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn" event={"ID":"52f56e47-435a-44dc-be93-b6cd22b2ce2e","Type":"ContainerStarted","Data":"29c2c2899b32a319a368c17c4486be6dbd6455f9a87ddd2f5dd7918ea9674828"}
Sep 18 06:46:42.301196 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:42.301152    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn" podStartSLOduration=1.301137755 podStartE2EDuration="1.301137755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:46:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:46:42.299449768 +0000 UTC m=+1467.980977513" watchObservedRunningTime="2026-09-18 06:46:42.301137755 +0000 UTC m=+1467.982665498"
Sep 18 06:46:42.445863 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:42.445826    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:42.450913 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:42.450888    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:42.543539 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:42.543461    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:43.287543 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:43.287511    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:43.288646 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:43.288623    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:46:43.592751 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:43.592663    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:44.233425 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:44.233393    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:44.334690 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:44.334649    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:44.920866 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:44.920835    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:45.643940 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:45.643909    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:46.330239 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:46.330186    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:47.047158 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:47.047125    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:47.575265 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:47.575193    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:47.662855 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:47.662821    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:47.726652 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:47.726624    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:47.787332 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:47.787300    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:47.855710 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:47.855622    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:47.907370 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:47.907342    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:48.102886 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:48.102849    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:48.203513 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:48.203480    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:48.327651 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:48.327621    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:48.415400 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:48.415369    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:48.696373 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:48.696337    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:49.409087 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:49.409050    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:49.593131 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:49.593097    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:46:50.168108 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:50.168076    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:50.810511 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:50.810483    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:51.529078 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:51.529046    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:52.234854 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:52.234815    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:52.837439 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:52.837403    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:54.132440 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:54.132397    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:54.188941 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:54.188912    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:54.382786 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:54.382709    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:54.790718 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:54.790686    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:55.077773 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:55.077695    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:55.252666 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:55.252630    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:56.295977 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:56.295946    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:56.392451 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:56.392421    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:46:56.492864 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:46:56.492834    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:08.976287 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:08.976249    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:12.794929 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:12.794868    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:14.153169 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:14.153137    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:14.399856 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:14.399820    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:14.858239 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:14.858135    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7753-9bfaa1"]
Sep 18 06:47:14.861882 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:14.861865    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-9bfaa1"
Sep 18 06:47:14.864372 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:14.864353    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-h78xn\""
Sep 18 06:47:14.870068 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:14.869642    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7753-9bfaa1"]
Sep 18 06:47:14.919379 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:14.919344    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgqgn\" (UniqueName: \"kubernetes.io/projected/2376e105-5adf-439a-a500-2b45e7912a6f-kube-api-access-pgqgn\") pod \"test-curl-7753-9bfaa1\" (UID: \"2376e105-5adf-439a-a500-2b45e7912a6f\") " pod="opendatahub/test-curl-7753-9bfaa1"
Sep 18 06:47:15.020803 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:15.020758    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgqgn\" (UniqueName: \"kubernetes.io/projected/2376e105-5adf-439a-a500-2b45e7912a6f-kube-api-access-pgqgn\") pod \"test-curl-7753-9bfaa1\" (UID: \"2376e105-5adf-439a-a500-2b45e7912a6f\") " pod="opendatahub/test-curl-7753-9bfaa1"
Sep 18 06:47:15.034572 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:15.034538    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgqgn\" (UniqueName: \"kubernetes.io/projected/2376e105-5adf-439a-a500-2b45e7912a6f-kube-api-access-pgqgn\") pod \"test-curl-7753-9bfaa1\" (UID: \"2376e105-5adf-439a-a500-2b45e7912a6f\") " pod="opendatahub/test-curl-7753-9bfaa1"
Sep 18 06:47:15.101394 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:15.101356    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:15.173606 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:15.173572    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-9bfaa1"
Sep 18 06:47:15.446773 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:15.446742    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7753-9bfaa1"]
Sep 18 06:47:15.447295 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:47:15.447265    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2376e105_5adf_439a_a500_2b45e7912a6f.slice/crio-e28e0cb41b48fa2f752df8874993c8ec2c9d33d3128fd0be9673253d97a4b9f0 WatchSource:0}: Error finding container e28e0cb41b48fa2f752df8874993c8ec2c9d33d3128fd0be9673253d97a4b9f0: Status 404 returned error can't find the container with id e28e0cb41b48fa2f752df8874993c8ec2c9d33d3128fd0be9673253d97a4b9f0
Sep 18 06:47:15.479827 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:15.479792    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7753-9bfaa1" event={"ID":"2376e105-5adf-439a-a500-2b45e7912a6f","Type":"ContainerStarted","Data":"e28e0cb41b48fa2f752df8874993c8ec2c9d33d3128fd0be9673253d97a4b9f0"}
Sep 18 06:47:15.791778 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:15.791676    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:16.413349 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:16.413306    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:16.488616 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:16.488578    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7753-9bfaa1" event={"ID":"2376e105-5adf-439a-a500-2b45e7912a6f","Type":"ContainerStarted","Data":"4ad429402414c2d5e208c6c1bbd677ae05193a047eaba31d814b230ca5517127"}
Sep 18 06:47:16.503274 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:16.503190    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7753-9bfaa1" podStartSLOduration=2.503170151 podStartE2EDuration="2.503170151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:47:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:47:16.501343774 +0000 UTC m=+1502.182871522" watchObservedRunningTime="2026-09-18 06:47:16.503170151 +0000 UTC m=+1502.184697897"
Sep 18 06:47:17.073326 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:17.073290    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7753-9bfaa1"]
Sep 18 06:47:17.077276 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:17.077250    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7753-9bfaa1"]
Sep 18 06:47:17.135795 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:17.135749    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:17.192007 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:17.191975    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7753-cecb6a"]
Sep 18 06:47:17.192452 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:17.192438    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2376e105-5adf-439a-a500-2b45e7912a6f" containerName="test-curl-7753-9bfaa1"
Sep 18 06:47:17.192452 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:17.192452    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2376e105-5adf-439a-a500-2b45e7912a6f" containerName="test-curl-7753-9bfaa1"
Sep 18 06:47:17.192546 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:17.192527    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2376e105-5adf-439a-a500-2b45e7912a6f" containerName="test-curl-7753-9bfaa1"
Sep 18 06:47:17.198901 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:17.198876    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-cecb6a"
Sep 18 06:47:17.199326 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:17.199305    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7753-cecb6a"]
Sep 18 06:47:17.250706 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:17.250676    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzq4r\" (UniqueName: \"kubernetes.io/projected/0ceac769-147c-4ca2-9349-b6a0f2c51a5a-kube-api-access-lzq4r\") pod \"test-curl-7753-cecb6a\" (UID: \"0ceac769-147c-4ca2-9349-b6a0f2c51a5a\") " pod="opendatahub/test-curl-7753-cecb6a"
Sep 18 06:47:17.351356 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:17.351328    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzq4r\" (UniqueName: \"kubernetes.io/projected/0ceac769-147c-4ca2-9349-b6a0f2c51a5a-kube-api-access-lzq4r\") pod \"test-curl-7753-cecb6a\" (UID: \"0ceac769-147c-4ca2-9349-b6a0f2c51a5a\") " pod="opendatahub/test-curl-7753-cecb6a"
Sep 18 06:47:17.358496 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:17.358467    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzq4r\" (UniqueName: \"kubernetes.io/projected/0ceac769-147c-4ca2-9349-b6a0f2c51a5a-kube-api-access-lzq4r\") pod \"test-curl-7753-cecb6a\" (UID: \"0ceac769-147c-4ca2-9349-b6a0f2c51a5a\") " pod="opendatahub/test-curl-7753-cecb6a"
Sep 18 06:47:17.376967 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:47:17.376944    2568 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/system-systemd\\\\x2dcoredump.slice/systemd-coredump@5-17833-0.service\": RecentStats: unable to find data in memory cache]"
Sep 18 06:47:17.495068 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:17.494987    2568 generic.go:356] "Generic (PLEG): container finished" podID="52f56e47-435a-44dc-be93-b6cd22b2ce2e" containerID="fb43b75821a5e34c7308716646637cf81fae806601a9489e52cf2c65f92088aa" exitCode=0
Sep 18 06:47:17.495068 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:17.495061    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn" event={"ID":"52f56e47-435a-44dc-be93-b6cd22b2ce2e","Type":"ContainerDied","Data":"fb43b75821a5e34c7308716646637cf81fae806601a9489e52cf2c65f92088aa"}
Sep 18 06:47:17.495599 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:47:17.495403    2568 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7753-9bfaa1" container="test-curl-7753-9bfaa1"
Sep 18 06:47:17.495599 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:17.495479    2568 scope.go:117] "RemoveContainer" containerID="fb43b75821a5e34c7308716646637cf81fae806601a9489e52cf2c65f92088aa"
Sep 18 06:47:17.496570 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:17.496538    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:47:17.496629 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:17.496615    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:47:17.496667 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:17.496643    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:47:17.511896 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:17.511874    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-cecb6a"
Sep 18 06:47:17.794025 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:17.793994    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7753-cecb6a"]
Sep 18 06:47:17.794701 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:47:17.794677    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ceac769_147c_4ca2_9349_b6a0f2c51a5a.slice/crio-973a29ec6aa16bebb2f184ec58dd531790e6fc79e6f69ba2f02d05170583c162 WatchSource:0}: Error finding container 973a29ec6aa16bebb2f184ec58dd531790e6fc79e6f69ba2f02d05170583c162: Status 404 returned error can't find the container with id 973a29ec6aa16bebb2f184ec58dd531790e6fc79e6f69ba2f02d05170583c162
Sep 18 06:47:17.852508 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:17.852483    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:18.456841 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:18.456807    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:18.502547 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:18.502501    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7753-cecb6a" event={"ID":"0ceac769-147c-4ca2-9349-b6a0f2c51a5a","Type":"ContainerStarted","Data":"f73ad9880dec010296d91aec15c51c60d16d7c08bb038e9efc8f48f0359b17e3"}
Sep 18 06:47:18.502973 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:18.502554    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7753-cecb6a" event={"ID":"0ceac769-147c-4ca2-9349-b6a0f2c51a5a","Type":"ContainerStarted","Data":"973a29ec6aa16bebb2f184ec58dd531790e6fc79e6f69ba2f02d05170583c162"}
Sep 18 06:47:18.504427 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:18.504399    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn" event={"ID":"52f56e47-435a-44dc-be93-b6cd22b2ce2e","Type":"ContainerStarted","Data":"6c959d28d5a56e6e156593ebe172bcedaf5f519c023e191a48cf20651b176e15"}
Sep 18 06:47:18.504557 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:18.504423    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7753-9bfaa1" podUID="2376e105-5adf-439a-a500-2b45e7912a6f" containerName="test-curl-7753-9bfaa1" containerID="cri-o://4ad429402414c2d5e208c6c1bbd677ae05193a047eaba31d814b230ca5517127" gracePeriod=2
Sep 18 06:47:18.518055 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:18.518003    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7753-cecb6a" podStartSLOduration=1.517985777 podStartE2EDuration="1.517985777s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:47:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:47:18.515328376 +0000 UTC m=+1504.196856125" watchObservedRunningTime="2026-09-18 06:47:18.517985777 +0000 UTC m=+1504.199513524"
Sep 18 06:47:18.533946 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:47:18.533904    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7753-9bfaa1\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="2376e105-5adf-439a-a500-2b45e7912a6f" pod="opendatahub/test-curl-7753-9bfaa1"
Sep 18 06:47:19.103326 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:19.103296    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7753-cecb6a"]
Sep 18 06:47:19.105848 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:19.105817    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7753-cecb6a"]
Sep 18 06:47:19.214599 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:19.214564    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7753-c85244"]
Sep 18 06:47:19.215035 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:19.215021    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ceac769-147c-4ca2-9349-b6a0f2c51a5a" containerName="test-curl-7753-cecb6a"
Sep 18 06:47:19.215082 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:19.215037    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ceac769-147c-4ca2-9349-b6a0f2c51a5a" containerName="test-curl-7753-cecb6a"
Sep 18 06:47:19.215117 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:19.215110    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ceac769-147c-4ca2-9349-b6a0f2c51a5a" containerName="test-curl-7753-cecb6a"
Sep 18 06:47:19.217330 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:19.217311    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-c85244"
Sep 18 06:47:19.223450 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:19.223426    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7753-c85244"]
Sep 18 06:47:19.271348 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:19.271311    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nx95\" (UniqueName: \"kubernetes.io/projected/d39d71b6-0bad-475b-a7e8-9c531fae0bbc-kube-api-access-8nx95\") pod \"test-curl-7753-c85244\" (UID: \"d39d71b6-0bad-475b-a7e8-9c531fae0bbc\") " pod="opendatahub/test-curl-7753-c85244"
Sep 18 06:47:19.372827 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:19.372778    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nx95\" (UniqueName: \"kubernetes.io/projected/d39d71b6-0bad-475b-a7e8-9c531fae0bbc-kube-api-access-8nx95\") pod \"test-curl-7753-c85244\" (UID: \"d39d71b6-0bad-475b-a7e8-9c531fae0bbc\") " pod="opendatahub/test-curl-7753-c85244"
Sep 18 06:47:19.380295 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:19.380266    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nx95\" (UniqueName: \"kubernetes.io/projected/d39d71b6-0bad-475b-a7e8-9c531fae0bbc-kube-api-access-8nx95\") pod \"test-curl-7753-c85244\" (UID: \"d39d71b6-0bad-475b-a7e8-9c531fae0bbc\") " pod="opendatahub/test-curl-7753-c85244"
Sep 18 06:47:19.445859 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:19.445819    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:47:19.446298 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:19.446264    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 18 06:47:19.446441 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:19.446334    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn" podUID="52f56e47-435a-44dc-be93-b6cd22b2ce2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 18 06:47:19.510257 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:47:19.510164    2568 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7753-cecb6a" container="test-curl-7753-cecb6a"
Sep 18 06:47:19.528972 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:19.528946    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-c85244"
Sep 18 06:47:19.817870 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:19.817840    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7753-c85244"]
Sep 18 06:47:19.819206 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:47:19.819171    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd39d71b6_0bad_475b_a7e8_9c531fae0bbc.slice/crio-b75b6d2b2c3157c1137065da3cdacb2201473996de6b8ed89de13668759d1fc2 WatchSource:0}: Error finding container b75b6d2b2c3157c1137065da3cdacb2201473996de6b8ed89de13668759d1fc2: Status 404 returned error can't find the container with id b75b6d2b2c3157c1137065da3cdacb2201473996de6b8ed89de13668759d1fc2
Sep 18 06:47:20.446270 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:20.446208    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 18 06:47:20.446489 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:20.446294    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn" podUID="52f56e47-435a-44dc-be93-b6cd22b2ce2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 18 06:47:20.519110 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:20.519074    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7753-c85244" event={"ID":"d39d71b6-0bad-475b-a7e8-9c531fae0bbc","Type":"ContainerStarted","Data":"68fb41e4389de180986ee189ae1bd25d39baab42b4b8388f295221743c9612b3"}
Sep 18 06:47:20.519511 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:20.519125    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7753-c85244" event={"ID":"d39d71b6-0bad-475b-a7e8-9c531fae0bbc","Type":"ContainerStarted","Data":"b75b6d2b2c3157c1137065da3cdacb2201473996de6b8ed89de13668759d1fc2"}
Sep 18 06:47:20.519511 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:20.519171    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7753-cecb6a" podUID="0ceac769-147c-4ca2-9349-b6a0f2c51a5a" containerName="test-curl-7753-cecb6a" containerID="cri-o://f73ad9880dec010296d91aec15c51c60d16d7c08bb038e9efc8f48f0359b17e3" gracePeriod=2
Sep 18 06:47:20.545241 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:47:20.543466    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7753-cecb6a\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="0ceac769-147c-4ca2-9349-b6a0f2c51a5a" pod="opendatahub/test-curl-7753-cecb6a"
Sep 18 06:47:20.549238 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:20.546122    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7753-c85244" podStartSLOduration=1.546101689 podStartE2EDuration="1.546101689s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:47:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:47:20.541400641 +0000 UTC m=+1506.222928387" watchObservedRunningTime="2026-09-18 06:47:20.546101689 +0000 UTC m=+1506.227629434"
Sep 18 06:47:20.549238 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:47:20.547048    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7753-cecb6a\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="0ceac769-147c-4ca2-9349-b6a0f2c51a5a" pod="opendatahub/test-curl-7753-cecb6a"
Sep 18 06:47:20.693169 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:20.693147    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-9bfaa1"
Sep 18 06:47:20.784573 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:20.784480    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgqgn\" (UniqueName: \"kubernetes.io/projected/2376e105-5adf-439a-a500-2b45e7912a6f-kube-api-access-pgqgn\") pod \"2376e105-5adf-439a-a500-2b45e7912a6f\" (UID: \"2376e105-5adf-439a-a500-2b45e7912a6f\") "
Sep 18 06:47:20.786397 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:20.786372    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2376e105-5adf-439a-a500-2b45e7912a6f-kube-api-access-pgqgn" (OuterVolumeSpecName: "kube-api-access-pgqgn") pod "2376e105-5adf-439a-a500-2b45e7912a6f" (UID: "2376e105-5adf-439a-a500-2b45e7912a6f"). InnerVolumeSpecName "kube-api-access-pgqgn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:47:20.885712 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:20.885672    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgqgn\" (UniqueName: \"kubernetes.io/projected/2376e105-5adf-439a-a500-2b45e7912a6f-kube-api-access-pgqgn\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:47:21.119517 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:21.119436    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7753-c85244"]
Sep 18 06:47:21.123274 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:21.123247    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7753-c85244"]
Sep 18 06:47:21.236413 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:21.236380    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7753-0cf5e6"]
Sep 18 06:47:21.237357 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:21.237331    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d39d71b6-0bad-475b-a7e8-9c531fae0bbc" containerName="test-curl-7753-c85244"
Sep 18 06:47:21.237521 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:21.237508    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d39d71b6-0bad-475b-a7e8-9c531fae0bbc" containerName="test-curl-7753-c85244"
Sep 18 06:47:21.237741 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:21.237727    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d39d71b6-0bad-475b-a7e8-9c531fae0bbc" containerName="test-curl-7753-c85244"
Sep 18 06:47:21.242029 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:21.242000    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-0cf5e6"
Sep 18 06:47:21.245208 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:21.245186    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7753-0cf5e6"]
Sep 18 06:47:21.290045 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:21.290008    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2tcr\" (UniqueName: \"kubernetes.io/projected/c420d568-7180-486a-9c7d-af11ca1e7923-kube-api-access-z2tcr\") pod \"test-curl-7753-0cf5e6\" (UID: \"c420d568-7180-486a-9c7d-af11ca1e7923\") " pod="opendatahub/test-curl-7753-0cf5e6"
Sep 18 06:47:21.390830 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:21.390746    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2tcr\" (UniqueName: \"kubernetes.io/projected/c420d568-7180-486a-9c7d-af11ca1e7923-kube-api-access-z2tcr\") pod \"test-curl-7753-0cf5e6\" (UID: \"c420d568-7180-486a-9c7d-af11ca1e7923\") " pod="opendatahub/test-curl-7753-0cf5e6"
Sep 18 06:47:21.399211 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:21.399178    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2tcr\" (UniqueName: \"kubernetes.io/projected/c420d568-7180-486a-9c7d-af11ca1e7923-kube-api-access-z2tcr\") pod \"test-curl-7753-0cf5e6\" (UID: \"c420d568-7180-486a-9c7d-af11ca1e7923\") " pod="opendatahub/test-curl-7753-0cf5e6"
Sep 18 06:47:21.446555 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:21.446495    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 18 06:47:21.446850 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:21.446809    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn" podUID="52f56e47-435a-44dc-be93-b6cd22b2ce2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 18 06:47:21.525723 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:21.525690    2568 generic.go:356] "Generic (PLEG): container finished" podID="2376e105-5adf-439a-a500-2b45e7912a6f" containerID="4ad429402414c2d5e208c6c1bbd677ae05193a047eaba31d814b230ca5517127" exitCode=137
Sep 18 06:47:21.526188 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:21.525730    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-9bfaa1"
Sep 18 06:47:21.526188 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:21.525742    2568 scope.go:117] "RemoveContainer" containerID="4ad429402414c2d5e208c6c1bbd677ae05193a047eaba31d814b230ca5517127"
Sep 18 06:47:21.526188 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:47:21.526043    2568 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7753-c85244" container="test-curl-7753-c85244"
Sep 18 06:47:21.551241 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:47:21.550411    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7753-9bfaa1\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="2376e105-5adf-439a-a500-2b45e7912a6f" pod="opendatahub/test-curl-7753-9bfaa1"
Sep 18 06:47:21.555375 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:21.555344    2568 scope.go:117] "RemoveContainer" containerID="4ad429402414c2d5e208c6c1bbd677ae05193a047eaba31d814b230ca5517127"
Sep 18 06:47:21.560239 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:47:21.557851    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ad429402414c2d5e208c6c1bbd677ae05193a047eaba31d814b230ca5517127\": container with ID starting with 4ad429402414c2d5e208c6c1bbd677ae05193a047eaba31d814b230ca5517127 not found: ID does not exist" containerID="4ad429402414c2d5e208c6c1bbd677ae05193a047eaba31d814b230ca5517127"
Sep 18 06:47:21.560239 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:21.557890    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ad429402414c2d5e208c6c1bbd677ae05193a047eaba31d814b230ca5517127"} err="failed to get container status \"4ad429402414c2d5e208c6c1bbd677ae05193a047eaba31d814b230ca5517127\": rpc error: code = NotFound desc = could not find container \"4ad429402414c2d5e208c6c1bbd677ae05193a047eaba31d814b230ca5517127\": container with ID starting with 4ad429402414c2d5e208c6c1bbd677ae05193a047eaba31d814b230ca5517127 not found: ID does not exist"
Sep 18 06:47:21.573612 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:21.573589    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-0cf5e6"
Sep 18 06:47:21.843241 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:21.842636    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7753-0cf5e6"]
Sep 18 06:47:22.202470 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:22.202433    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:47:22.446516 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:22.446479    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 18 06:47:22.446769 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:22.446738    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn" podUID="52f56e47-435a-44dc-be93-b6cd22b2ce2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 18 06:47:22.536201 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:22.536161    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7753-0cf5e6" event={"ID":"c420d568-7180-486a-9c7d-af11ca1e7923","Type":"ContainerStarted","Data":"86f2da76138705e9f6df57be42121417f19cc23e2a4aa856ba54bb9bcffd71e1"}
Sep 18 06:47:22.536631 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:22.536232    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7753-0cf5e6" event={"ID":"c420d568-7180-486a-9c7d-af11ca1e7923","Type":"ContainerStarted","Data":"1f8f70c05b02ba3b4fd9a29e9ddae69abf2dffd262a47ba284f975ab519d1403"}
Sep 18 06:47:22.539024 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:47:22.538875    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7753-9bfaa1\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="2376e105-5adf-439a-a500-2b45e7912a6f" pod="opendatahub/test-curl-7753-9bfaa1"
Sep 18 06:47:22.549641 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:22.549329    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7753-c85244" podUID="d39d71b6-0bad-475b-a7e8-9c531fae0bbc" containerName="test-curl-7753-c85244" containerID="cri-o://68fb41e4389de180986ee189ae1bd25d39baab42b4b8388f295221743c9612b3" gracePeriod=2
Sep 18 06:47:22.554249 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:22.553580    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2376e105-5adf-439a-a500-2b45e7912a6f" path="/var/lib/kubelet/pods/2376e105-5adf-439a-a500-2b45e7912a6f/volumes"
Sep 18 06:47:22.710828 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:22.710803    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-cecb6a"
Sep 18 06:47:22.806345 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:22.806273    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzq4r\" (UniqueName: \"kubernetes.io/projected/0ceac769-147c-4ca2-9349-b6a0f2c51a5a-kube-api-access-lzq4r\") pod \"0ceac769-147c-4ca2-9349-b6a0f2c51a5a\" (UID: \"0ceac769-147c-4ca2-9349-b6a0f2c51a5a\") "
Sep 18 06:47:22.808183 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:22.808164    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ceac769-147c-4ca2-9349-b6a0f2c51a5a-kube-api-access-lzq4r" (OuterVolumeSpecName: "kube-api-access-lzq4r") pod "0ceac769-147c-4ca2-9349-b6a0f2c51a5a" (UID: "0ceac769-147c-4ca2-9349-b6a0f2c51a5a"). InnerVolumeSpecName "kube-api-access-lzq4r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:47:22.907687 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:22.907654    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzq4r\" (UniqueName: \"kubernetes.io/projected/0ceac769-147c-4ca2-9349-b6a0f2c51a5a-kube-api-access-lzq4r\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:47:23.133868 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:23.133741    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7753-0cf5e6" podStartSLOduration=2.133715147 podStartE2EDuration="2.133715147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:47:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:47:22.558625045 +0000 UTC m=+1508.240152793" watchObservedRunningTime="2026-09-18 06:47:23.133715147 +0000 UTC m=+1508.815242892"
Sep 18 06:47:23.134322 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:23.134296    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7753-0cf5e6"]
Sep 18 06:47:23.140062 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:23.140032    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7753-0cf5e6"]
Sep 18 06:47:23.446719 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:23.446677    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 18 06:47:23.447008 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:23.446978    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn" podUID="52f56e47-435a-44dc-be93-b6cd22b2ce2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 18 06:47:23.561151 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:23.561113    2568 generic.go:356] "Generic (PLEG): container finished" podID="0ceac769-147c-4ca2-9349-b6a0f2c51a5a" containerID="f73ad9880dec010296d91aec15c51c60d16d7c08bb038e9efc8f48f0359b17e3" exitCode=137
Sep 18 06:47:23.561665 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:23.561176    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-cecb6a"
Sep 18 06:47:23.561665 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:23.561232    2568 scope.go:117] "RemoveContainer" containerID="f73ad9880dec010296d91aec15c51c60d16d7c08bb038e9efc8f48f0359b17e3"
Sep 18 06:47:23.561665 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:47:23.561504    2568 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7753-0cf5e6" container="test-curl-7753-0cf5e6"
Sep 18 06:47:23.571121 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:23.571105    2568 scope.go:117] "RemoveContainer" containerID="f73ad9880dec010296d91aec15c51c60d16d7c08bb038e9efc8f48f0359b17e3"
Sep 18 06:47:23.571441 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:47:23.571417    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f73ad9880dec010296d91aec15c51c60d16d7c08bb038e9efc8f48f0359b17e3\": container with ID starting with f73ad9880dec010296d91aec15c51c60d16d7c08bb038e9efc8f48f0359b17e3 not found: ID does not exist" containerID="f73ad9880dec010296d91aec15c51c60d16d7c08bb038e9efc8f48f0359b17e3"
Sep 18 06:47:23.571499 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:23.571449    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f73ad9880dec010296d91aec15c51c60d16d7c08bb038e9efc8f48f0359b17e3"} err="failed to get container status \"f73ad9880dec010296d91aec15c51c60d16d7c08bb038e9efc8f48f0359b17e3\": rpc error: code = NotFound desc = could not find container \"f73ad9880dec010296d91aec15c51c60d16d7c08bb038e9efc8f48f0359b17e3\": container with ID starting with f73ad9880dec010296d91aec15c51c60d16d7c08bb038e9efc8f48f0359b17e3 not found: ID does not exist"
Sep 18 06:47:23.571751 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:47:23.571731    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7753-cecb6a\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="0ceac769-147c-4ca2-9349-b6a0f2c51a5a" pod="opendatahub/test-curl-7753-cecb6a"
Sep 18 06:47:23.695958 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:23.695928    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:23.971242 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:23.970834    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:24.342246 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:24.342125    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:24.439835 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:24.439797    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:24.450248 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:24.450198    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:47:24.450608 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:24.450591    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:47:24.451341 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:24.451325    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:47:24.472964 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:47:24.472917    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7753-cecb6a\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="0ceac769-147c-4ca2-9349-b6a0f2c51a5a" pod="opendatahub/test-curl-7753-cecb6a"
Sep 18 06:47:24.494570 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:47:24.494534    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7753-cecb6a\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="0ceac769-147c-4ca2-9349-b6a0f2c51a5a" pod="opendatahub/test-curl-7753-cecb6a"
Sep 18 06:47:24.545153 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:24.545122    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ceac769-147c-4ca2-9349-b6a0f2c51a5a" path="/var/lib/kubelet/pods/0ceac769-147c-4ca2-9349-b6a0f2c51a5a/volumes"
Sep 18 06:47:24.569328 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:24.569299    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7753-0cf5e6" podUID="c420d568-7180-486a-9c7d-af11ca1e7923" containerName="test-curl-7753-0cf5e6" containerID="cri-o://86f2da76138705e9f6df57be42121417f19cc23e2a4aa856ba54bb9bcffd71e1" gracePeriod=2
Sep 18 06:47:24.700620 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:24.700595    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-c85244"
Sep 18 06:47:24.825629 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:24.825588    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nx95\" (UniqueName: \"kubernetes.io/projected/d39d71b6-0bad-475b-a7e8-9c531fae0bbc-kube-api-access-8nx95\") pod \"d39d71b6-0bad-475b-a7e8-9c531fae0bbc\" (UID: \"d39d71b6-0bad-475b-a7e8-9c531fae0bbc\") "
Sep 18 06:47:24.827754 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:24.827718    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d39d71b6-0bad-475b-a7e8-9c531fae0bbc-kube-api-access-8nx95" (OuterVolumeSpecName: "kube-api-access-8nx95") pod "d39d71b6-0bad-475b-a7e8-9c531fae0bbc" (UID: "d39d71b6-0bad-475b-a7e8-9c531fae0bbc"). InnerVolumeSpecName "kube-api-access-8nx95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:47:24.926576 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:24.926542    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nx95\" (UniqueName: \"kubernetes.io/projected/d39d71b6-0bad-475b-a7e8-9c531fae0bbc-kube-api-access-8nx95\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:47:25.121958 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:25.121922    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:25.575955 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:25.575918    2568 generic.go:356] "Generic (PLEG): container finished" podID="d39d71b6-0bad-475b-a7e8-9c531fae0bbc" containerID="68fb41e4389de180986ee189ae1bd25d39baab42b4b8388f295221743c9612b3" exitCode=137
Sep 18 06:47:25.576445 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:25.575972    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-c85244"
Sep 18 06:47:25.576445 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:25.576028    2568 scope.go:117] "RemoveContainer" containerID="68fb41e4389de180986ee189ae1bd25d39baab42b4b8388f295221743c9612b3"
Sep 18 06:47:25.585480 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:25.585463    2568 scope.go:117] "RemoveContainer" containerID="68fb41e4389de180986ee189ae1bd25d39baab42b4b8388f295221743c9612b3"
Sep 18 06:47:25.585740 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:47:25.585721    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68fb41e4389de180986ee189ae1bd25d39baab42b4b8388f295221743c9612b3\": container with ID starting with 68fb41e4389de180986ee189ae1bd25d39baab42b4b8388f295221743c9612b3 not found: ID does not exist" containerID="68fb41e4389de180986ee189ae1bd25d39baab42b4b8388f295221743c9612b3"
Sep 18 06:47:25.585791 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:25.585750    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68fb41e4389de180986ee189ae1bd25d39baab42b4b8388f295221743c9612b3"} err="failed to get container status \"68fb41e4389de180986ee189ae1bd25d39baab42b4b8388f295221743c9612b3\": rpc error: code = NotFound desc = could not find container \"68fb41e4389de180986ee189ae1bd25d39baab42b4b8388f295221743c9612b3\": container with ID starting with 68fb41e4389de180986ee189ae1bd25d39baab42b4b8388f295221743c9612b3 not found: ID does not exist"
Sep 18 06:47:25.586044 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:47:25.586024    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7753-c85244\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="d39d71b6-0bad-475b-a7e8-9c531fae0bbc" pod="opendatahub/test-curl-7753-c85244"
Sep 18 06:47:25.806204 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:25.806170    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:26.541201 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:47:26.541167    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7753-c85244\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="d39d71b6-0bad-475b-a7e8-9c531fae0bbc" pod="opendatahub/test-curl-7753-c85244"
Sep 18 06:47:26.543740 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:26.543712    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:26.544440 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:26.544422    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d39d71b6-0bad-475b-a7e8-9c531fae0bbc" path="/var/lib/kubelet/pods/d39d71b6-0bad-475b-a7e8-9c531fae0bbc/volumes"
Sep 18 06:47:26.762240 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:26.762203    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-0cf5e6"
Sep 18 06:47:26.848276 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:26.848176    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2tcr\" (UniqueName: \"kubernetes.io/projected/c420d568-7180-486a-9c7d-af11ca1e7923-kube-api-access-z2tcr\") pod \"c420d568-7180-486a-9c7d-af11ca1e7923\" (UID: \"c420d568-7180-486a-9c7d-af11ca1e7923\") "
Sep 18 06:47:26.850553 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:26.850521    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c420d568-7180-486a-9c7d-af11ca1e7923-kube-api-access-z2tcr" (OuterVolumeSpecName: "kube-api-access-z2tcr") pod "c420d568-7180-486a-9c7d-af11ca1e7923" (UID: "c420d568-7180-486a-9c7d-af11ca1e7923"). InnerVolumeSpecName "kube-api-access-z2tcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:47:26.949765 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:26.949737    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2tcr\" (UniqueName: \"kubernetes.io/projected/c420d568-7180-486a-9c7d-af11ca1e7923-kube-api-access-z2tcr\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:47:27.086780 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:27.086698    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:27.186459 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:27.186428    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:27.402071 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:27.402039    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:27.589728 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:27.589650    2568 generic.go:356] "Generic (PLEG): container finished" podID="c420d568-7180-486a-9c7d-af11ca1e7923" containerID="86f2da76138705e9f6df57be42121417f19cc23e2a4aa856ba54bb9bcffd71e1" exitCode=137
Sep 18 06:47:27.589728 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:27.589688    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7753-0cf5e6"
Sep 18 06:47:27.589922 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:27.589748    2568 scope.go:117] "RemoveContainer" containerID="86f2da76138705e9f6df57be42121417f19cc23e2a4aa856ba54bb9bcffd71e1"
Sep 18 06:47:27.598925 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:27.598905    2568 scope.go:117] "RemoveContainer" containerID="86f2da76138705e9f6df57be42121417f19cc23e2a4aa856ba54bb9bcffd71e1"
Sep 18 06:47:27.599154 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:47:27.599137    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86f2da76138705e9f6df57be42121417f19cc23e2a4aa856ba54bb9bcffd71e1\": container with ID starting with 86f2da76138705e9f6df57be42121417f19cc23e2a4aa856ba54bb9bcffd71e1 not found: ID does not exist" containerID="86f2da76138705e9f6df57be42121417f19cc23e2a4aa856ba54bb9bcffd71e1"
Sep 18 06:47:27.599260 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:27.599161    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86f2da76138705e9f6df57be42121417f19cc23e2a4aa856ba54bb9bcffd71e1"} err="failed to get container status \"86f2da76138705e9f6df57be42121417f19cc23e2a4aa856ba54bb9bcffd71e1\": rpc error: code = NotFound desc = could not find container \"86f2da76138705e9f6df57be42121417f19cc23e2a4aa856ba54bb9bcffd71e1\": container with ID starting with 86f2da76138705e9f6df57be42121417f19cc23e2a4aa856ba54bb9bcffd71e1 not found: ID does not exist"
Sep 18 06:47:27.599813 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:47:27.599793    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7753-0cf5e6\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="c420d568-7180-486a-9c7d-af11ca1e7923" pod="opendatahub/test-curl-7753-0cf5e6"
Sep 18 06:47:28.032829 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:28.032797    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:28.544014 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:28.543981    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c420d568-7180-486a-9c7d-af11ca1e7923" path="/var/lib/kubelet/pods/c420d568-7180-486a-9c7d-af11ca1e7923/volumes"
Sep 18 06:47:28.826087 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:28.825997    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:29.539823 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:29.539783    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:30.337398 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:30.337361    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:30.438723 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:30.438689    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:30.688210 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:30.688174    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:30.792036 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:30.792004    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:30.880483 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:30.880447    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:31.153731 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:31.153700    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:31.947745 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:31.947686    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:32.651196 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:32.651168    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:34.242302 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:34.242272    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:34.773089 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:34.773050    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7c4f4b-29828565-f84cn"]
Sep 18 06:47:34.775275 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:34.775251    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-a-7c4f4b-29828565-f84cn"]
Sep 18 06:47:35.354548 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:35.354515    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:35.457288 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:35.457250    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:36.041402 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:36.041339    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:36.545617 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:36.545584    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28ce143b-6486-4db0-b395-56c03da9e731" path="/var/lib/kubelet/pods/28ce143b-6486-4db0-b395-56c03da9e731/volumes"
Sep 18 06:47:36.664252 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:36.664196    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:37.436004 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:37.435970    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:38.039299 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:38.039262    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:38.736653 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:38.736621    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:39.350363 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:39.350332    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:41.063341 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:41.063305    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:41.164463 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:41.164425    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:41.322921 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:41.322836    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:41.910031 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:41.910001    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:42.627429 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:42.627369    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:43.326863 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:43.326832    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:43.986200 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:43.986168    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:44.727229 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:44.727184    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:45.331134 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:45.331097    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:46.317660 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:46.317622    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:46.864442 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:46.864383    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:46.966789 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:46.966758    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:47.640879 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:47.640840    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:48.259729 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:48.259693    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:49.032936 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:49.032898    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:49.627700 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:49.627671    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:50.312140 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:50.312108    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:50.925070 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:50.925040    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:51.758798 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:51.758766    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:52.305384 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.305352    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"]
Sep 18 06:47:52.305834 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.305599    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf" podUID="88717d6b-2a8f-4023-a308-0ec483232deb" containerName="maas-api" containerID="cri-o://2ad9387f582d5f29a123085b2fd65ba18d05ed36bf091cf311d5af01d8bad82a" gracePeriod=30
Sep 18 06:47:52.332533 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.332504    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:52.427899 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.427872    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:52.509643 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.509611    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:52.565682 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.565615    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:52.571581 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.571562    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"
Sep 18 06:47:52.634846 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.634815    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88717d6b-2a8f-4023-a308-0ec483232deb-openshift-service-ca\") pod \"88717d6b-2a8f-4023-a308-0ec483232deb\" (UID: \"88717d6b-2a8f-4023-a308-0ec483232deb\") "
Sep 18 06:47:52.635170 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.635149    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blkps\" (UniqueName: \"kubernetes.io/projected/88717d6b-2a8f-4023-a308-0ec483232deb-kube-api-access-blkps\") pod \"88717d6b-2a8f-4023-a308-0ec483232deb\" (UID: \"88717d6b-2a8f-4023-a308-0ec483232deb\") "
Sep 18 06:47:52.635308 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.635161    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88717d6b-2a8f-4023-a308-0ec483232deb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "88717d6b-2a8f-4023-a308-0ec483232deb" (UID: "88717d6b-2a8f-4023-a308-0ec483232deb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 06:47:52.635453 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.635434    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88717d6b-2a8f-4023-a308-0ec483232deb-maas-api-tls\") pod \"88717d6b-2a8f-4023-a308-0ec483232deb\" (UID: \"88717d6b-2a8f-4023-a308-0ec483232deb\") "
Sep 18 06:47:52.635644 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.635629    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88717d6b-2a8f-4023-a308-0ec483232deb-metrics-certs\") pod \"88717d6b-2a8f-4023-a308-0ec483232deb\" (UID: \"88717d6b-2a8f-4023-a308-0ec483232deb\") "
Sep 18 06:47:52.636370 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.636346    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88717d6b-2a8f-4023-a308-0ec483232deb-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:47:52.637963 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.637930    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88717d6b-2a8f-4023-a308-0ec483232deb-kube-api-access-blkps" (OuterVolumeSpecName: "kube-api-access-blkps") pod "88717d6b-2a8f-4023-a308-0ec483232deb" (UID: "88717d6b-2a8f-4023-a308-0ec483232deb"). InnerVolumeSpecName "kube-api-access-blkps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:47:52.638165 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.638130    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88717d6b-2a8f-4023-a308-0ec483232deb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "88717d6b-2a8f-4023-a308-0ec483232deb" (UID: "88717d6b-2a8f-4023-a308-0ec483232deb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:47:52.639083 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.639047    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88717d6b-2a8f-4023-a308-0ec483232deb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "88717d6b-2a8f-4023-a308-0ec483232deb" (UID: "88717d6b-2a8f-4023-a308-0ec483232deb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:47:52.738742 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.738710    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/88717d6b-2a8f-4023-a308-0ec483232deb-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:47:52.738742 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.738739    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blkps\" (UniqueName: \"kubernetes.io/projected/88717d6b-2a8f-4023-a308-0ec483232deb-kube-api-access-blkps\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:47:52.738742 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.738750    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88717d6b-2a8f-4023-a308-0ec483232deb-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:47:52.746012 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.745985    2568 generic.go:356] "Generic (PLEG): container finished" podID="88717d6b-2a8f-4023-a308-0ec483232deb" containerID="2ad9387f582d5f29a123085b2fd65ba18d05ed36bf091cf311d5af01d8bad82a" exitCode=0
Sep 18 06:47:52.746143 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.746028    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"
Sep 18 06:47:52.746143 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.746033    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf" event={"ID":"88717d6b-2a8f-4023-a308-0ec483232deb","Type":"ContainerDied","Data":"2ad9387f582d5f29a123085b2fd65ba18d05ed36bf091cf311d5af01d8bad82a"}
Sep 18 06:47:52.746143 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.746077    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf" event={"ID":"88717d6b-2a8f-4023-a308-0ec483232deb","Type":"ContainerDied","Data":"82cd34cc552f691fc42f916fe7b73e462cdaca37a54986cfdfcab27c8da687e9"}
Sep 18 06:47:52.746143 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.746100    2568 scope.go:117] "RemoveContainer" containerID="2ad9387f582d5f29a123085b2fd65ba18d05ed36bf091cf311d5af01d8bad82a"
Sep 18 06:47:52.756122 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.756104    2568 scope.go:117] "RemoveContainer" containerID="2ad9387f582d5f29a123085b2fd65ba18d05ed36bf091cf311d5af01d8bad82a"
Sep 18 06:47:52.756442 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:47:52.756418    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ad9387f582d5f29a123085b2fd65ba18d05ed36bf091cf311d5af01d8bad82a\": container with ID starting with 2ad9387f582d5f29a123085b2fd65ba18d05ed36bf091cf311d5af01d8bad82a not found: ID does not exist" containerID="2ad9387f582d5f29a123085b2fd65ba18d05ed36bf091cf311d5af01d8bad82a"
Sep 18 06:47:52.756503 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.756454    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ad9387f582d5f29a123085b2fd65ba18d05ed36bf091cf311d5af01d8bad82a"} err="failed to get container status \"2ad9387f582d5f29a123085b2fd65ba18d05ed36bf091cf311d5af01d8bad82a\": rpc error: code = NotFound desc = could not find container \"2ad9387f582d5f29a123085b2fd65ba18d05ed36bf091cf311d5af01d8bad82a\": container with ID starting with 2ad9387f582d5f29a123085b2fd65ba18d05ed36bf091cf311d5af01d8bad82a not found: ID does not exist"
Sep 18 06:47:52.767662 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.767641    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"]
Sep 18 06:47:52.771108 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.771087    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-708b6f-5d7669f49c-pzdqf"]
Sep 18 06:47:52.841838 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.841770    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:52.942301 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:52.942257    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:53.627703 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:53.627671    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:53.920212 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:53.920179    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"]
Sep 18 06:47:53.984150 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:53.984091    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:54.359237 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:54.359139    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:54.544628 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:54.544594    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88717d6b-2a8f-4023-a308-0ec483232deb" path="/var/lib/kubelet/pods/88717d6b-2a8f-4023-a308-0ec483232deb/volumes"
Sep 18 06:47:55.012457 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:55.012421    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:55.741436 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:55.741405    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:56.351774 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:56.351742    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:47:57.918937 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:47:57.918905    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:03.290497 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:03.290462    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:48:03.880168 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:03.880134    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:03.981344 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:03.981315    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:08.939240 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:08.939169    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n" podUID="7ae10764-63e1-4de0-9961-ccb4c458e4d9" containerName="main" containerID="cri-o://ef88beba0afb920924574d5ecb21148ada76f6f6c152f4dbdff6f1e1b0846bdf" gracePeriod=45
Sep 18 06:48:09.192721 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.192663    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:48:09.318340 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.318295    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-home\") pod \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") "
Sep 18 06:48:09.318340 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.318342    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-model-cache\") pod \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") "
Sep 18 06:48:09.318601 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.318431    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ae10764-63e1-4de0-9961-ccb4c458e4d9-tls-certs\") pod \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") "
Sep 18 06:48:09.318601 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.318532    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-tmp-dir\") pod \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") "
Sep 18 06:48:09.318601 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.318591    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wj2z\" (UniqueName: \"kubernetes.io/projected/7ae10764-63e1-4de0-9961-ccb4c458e4d9-kube-api-access-7wj2z\") pod \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") "
Sep 18 06:48:09.318767 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.318630    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-dshm\") pod \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\" (UID: \"7ae10764-63e1-4de0-9961-ccb4c458e4d9\") "
Sep 18 06:48:09.318767 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.318632    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-home" (OuterVolumeSpecName: "home") pod "7ae10764-63e1-4de0-9961-ccb4c458e4d9" (UID: "7ae10764-63e1-4de0-9961-ccb4c458e4d9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:48:09.318767 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.318690    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-model-cache" (OuterVolumeSpecName: "model-cache") pod "7ae10764-63e1-4de0-9961-ccb4c458e4d9" (UID: "7ae10764-63e1-4de0-9961-ccb4c458e4d9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:48:09.318882 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.318810    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7ae10764-63e1-4de0-9961-ccb4c458e4d9" (UID: "7ae10764-63e1-4de0-9961-ccb4c458e4d9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:48:09.319065 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.319043    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:48:09.319156 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.319065    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:48:09.319156 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.319079    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:48:09.320813 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.320785    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ae10764-63e1-4de0-9961-ccb4c458e4d9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7ae10764-63e1-4de0-9961-ccb4c458e4d9" (UID: "7ae10764-63e1-4de0-9961-ccb4c458e4d9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:48:09.320931 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.320814    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-dshm" (OuterVolumeSpecName: "dshm") pod "7ae10764-63e1-4de0-9961-ccb4c458e4d9" (UID: "7ae10764-63e1-4de0-9961-ccb4c458e4d9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:48:09.320931 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.320860    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ae10764-63e1-4de0-9961-ccb4c458e4d9-kube-api-access-7wj2z" (OuterVolumeSpecName: "kube-api-access-7wj2z") pod "7ae10764-63e1-4de0-9961-ccb4c458e4d9" (UID: "7ae10764-63e1-4de0-9961-ccb4c458e4d9"). InnerVolumeSpecName "kube-api-access-7wj2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:48:09.420620 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.420593    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wj2z\" (UniqueName: \"kubernetes.io/projected/7ae10764-63e1-4de0-9961-ccb4c458e4d9-kube-api-access-7wj2z\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:48:09.420620 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.420614    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7ae10764-63e1-4de0-9961-ccb4c458e4d9-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:48:09.420620 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.420624    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7ae10764-63e1-4de0-9961-ccb4c458e4d9-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:48:09.845233 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.845182    2568 generic.go:356] "Generic (PLEG): container finished" podID="7ae10764-63e1-4de0-9961-ccb4c458e4d9" containerID="ef88beba0afb920924574d5ecb21148ada76f6f6c152f4dbdff6f1e1b0846bdf" exitCode=0
Sep 18 06:48:09.845425 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.845237    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"
Sep 18 06:48:09.845425 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.845267    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n" event={"ID":"7ae10764-63e1-4de0-9961-ccb4c458e4d9","Type":"ContainerDied","Data":"ef88beba0afb920924574d5ecb21148ada76f6f6c152f4dbdff6f1e1b0846bdf"}
Sep 18 06:48:09.845425 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.845313    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n" event={"ID":"7ae10764-63e1-4de0-9961-ccb4c458e4d9","Type":"ContainerDied","Data":"d3fe0386c9993e79d1361b95d29f7b27927fb50cb74708952e9273b6f0693bce"}
Sep 18 06:48:09.845425 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.845331    2568 scope.go:117] "RemoveContainer" containerID="ef88beba0afb920924574d5ecb21148ada76f6f6c152f4dbdff6f1e1b0846bdf"
Sep 18 06:48:09.855193 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.855179    2568 scope.go:117] "RemoveContainer" containerID="ef88beba0afb920924574d5ecb21148ada76f6f6c152f4dbdff6f1e1b0846bdf"
Sep 18 06:48:09.855462 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:48:09.855443    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef88beba0afb920924574d5ecb21148ada76f6f6c152f4dbdff6f1e1b0846bdf\": container with ID starting with ef88beba0afb920924574d5ecb21148ada76f6f6c152f4dbdff6f1e1b0846bdf not found: ID does not exist" containerID="ef88beba0afb920924574d5ecb21148ada76f6f6c152f4dbdff6f1e1b0846bdf"
Sep 18 06:48:09.855501 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.855472    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef88beba0afb920924574d5ecb21148ada76f6f6c152f4dbdff6f1e1b0846bdf"} err="failed to get container status \"ef88beba0afb920924574d5ecb21148ada76f6f6c152f4dbdff6f1e1b0846bdf\": rpc error: code = NotFound desc = could not find container \"ef88beba0afb920924574d5ecb21148ada76f6f6c152f4dbdff6f1e1b0846bdf\": container with ID starting with ef88beba0afb920924574d5ecb21148ada76f6f6c152f4dbdff6f1e1b0846bdf not found: ID does not exist"
Sep 18 06:48:09.866455 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.866435    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"]
Sep 18 06:48:09.871399 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:09.871379    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-708b6f/test-model-708b6f-kserve-67746c4b67-27n2n"]
Sep 18 06:48:10.544517 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:10.544478    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ae10764-63e1-4de0-9961-ccb4c458e4d9" path="/var/lib/kubelet/pods/7ae10764-63e1-4de0-9961-ccb4c458e4d9/volumes"
Sep 18 06:48:12.216433 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:12.216404    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:28.005919 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:28.005880    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:32.784665 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.784629    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"]
Sep 18 06:48:32.785304 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.785286    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88717d6b-2a8f-4023-a308-0ec483232deb" containerName="maas-api"
Sep 18 06:48:32.785376 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.785306    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="88717d6b-2a8f-4023-a308-0ec483232deb" containerName="maas-api"
Sep 18 06:48:32.785376 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.785320    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ae10764-63e1-4de0-9961-ccb4c458e4d9" containerName="main"
Sep 18 06:48:32.785376 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.785329    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ae10764-63e1-4de0-9961-ccb4c458e4d9" containerName="main"
Sep 18 06:48:32.785376 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.785366    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c420d568-7180-486a-9c7d-af11ca1e7923" containerName="test-curl-7753-0cf5e6"
Sep 18 06:48:32.785526 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.785377    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c420d568-7180-486a-9c7d-af11ca1e7923" containerName="test-curl-7753-0cf5e6"
Sep 18 06:48:32.785526 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.785482    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c420d568-7180-486a-9c7d-af11ca1e7923" containerName="test-curl-7753-0cf5e6"
Sep 18 06:48:32.785526 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.785493    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="88717d6b-2a8f-4023-a308-0ec483232deb" containerName="maas-api"
Sep 18 06:48:32.785526 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.785504    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ae10764-63e1-4de0-9961-ccb4c458e4d9" containerName="main"
Sep 18 06:48:32.790469 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.790451    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:32.794955 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.794441    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-93683709\"/\"kube-root-ca.crt\""
Sep 18 06:48:32.794955 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.794501    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-93683709\"/\"default-dockercfg-lrbjh\""
Sep 18 06:48:32.794955 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.794588    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-93683709\"/\"openshift-service-ca.crt\""
Sep 18 06:48:32.794955 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.794792    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-93683709\"/\"e2e-lifecycle-model-93683709-kserve-self-signed-certs\""
Sep 18 06:48:32.798009 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.797987    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"]
Sep 18 06:48:32.816474 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.816426    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:32.866172 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.866145    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-home\") pod \"e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") " pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:32.866172 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.866174    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-dshm\") pod \"e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") " pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:32.866363 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.866318    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xqfr\" (UniqueName: \"kubernetes.io/projected/f9699e03-05ac-4b5a-9555-52a5d39b62e3-kube-api-access-9xqfr\") pod \"e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") " pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:32.866363 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.866354    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9699e03-05ac-4b5a-9555-52a5d39b62e3-tls-certs\") pod \"e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") " pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:32.866451 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.866380    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-tmp-dir\") pod \"e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") " pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:32.866451 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.866426    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-model-cache\") pod \"e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") " pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:32.967611 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.967574    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-home\") pod \"e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") " pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:32.967611 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.967610    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-dshm\") pod \"e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") " pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:32.967875 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.967679    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xqfr\" (UniqueName: \"kubernetes.io/projected/f9699e03-05ac-4b5a-9555-52a5d39b62e3-kube-api-access-9xqfr\") pod \"e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") " pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:32.967875 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.967702    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9699e03-05ac-4b5a-9555-52a5d39b62e3-tls-certs\") pod \"e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") " pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:32.967875 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.967719    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-tmp-dir\") pod \"e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") " pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:32.967875 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.967745    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-model-cache\") pod \"e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") " pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:32.968093 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.968056    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-home\") pod \"e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") " pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:32.968136 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.968118    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-model-cache\") pod \"e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") " pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:32.968171 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.968142    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-tmp-dir\") pod \"e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") " pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:32.969983 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.969955    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-dshm\") pod \"e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") " pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:32.970117 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.970098    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9699e03-05ac-4b5a-9555-52a5d39b62e3-tls-certs\") pod \"e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") " pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:32.975493 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:32.975470    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xqfr\" (UniqueName: \"kubernetes.io/projected/f9699e03-05ac-4b5a-9555-52a5d39b62e3-kube-api-access-9xqfr\") pod \"e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") " pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:33.101960 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:33.101875    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:33.371535 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:33.371502    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"]
Sep 18 06:48:33.372275 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:48:33.372253    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9699e03_05ac_4b5a_9555_52a5d39b62e3.slice/crio-f5772dea7da20b2cfefe5ab5b6984d448a83ec2e719ad80a79825fec6514c216 WatchSource:0}: Error finding container f5772dea7da20b2cfefe5ab5b6984d448a83ec2e719ad80a79825fec6514c216: Status 404 returned error can't find the container with id f5772dea7da20b2cfefe5ab5b6984d448a83ec2e719ad80a79825fec6514c216
Sep 18 06:48:33.983412 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:33.983376    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b" event={"ID":"f9699e03-05ac-4b5a-9555-52a5d39b62e3","Type":"ContainerStarted","Data":"0ca6566058575d9d56fb741cc503f0cfe605aea078e1cbd385cff42df0dce63a"}
Sep 18 06:48:33.983773 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:33.983419    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b" event={"ID":"f9699e03-05ac-4b5a-9555-52a5d39b62e3","Type":"ContainerStarted","Data":"f5772dea7da20b2cfefe5ab5b6984d448a83ec2e719ad80a79825fec6514c216"}
Sep 18 06:48:34.000326 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:34.000280    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b" podStartSLOduration=2.000265276 podStartE2EDuration="2.000265276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:48:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:48:33.998669468 +0000 UTC m=+1579.680197204" watchObservedRunningTime="2026-09-18 06:48:34.000265276 +0000 UTC m=+1579.681793022"
Sep 18 06:48:34.660969 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:34.660938    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:34.879035 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:34.879003    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"]
Sep 18 06:48:34.879297 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:34.879269    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b" podUID="96c8cb03-28cf-41cb-829d-a7b9073bdb1c" containerName="istio-proxy" containerID="cri-o://9e7bccb2d87e1176789e1b18af025644227fdf29474633d9914d34a16d3c13c3" gracePeriod=30
Sep 18 06:48:34.894463 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:34.894437    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:35.432465 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.432427    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:35.683600 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.683526    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"]
Sep 18 06:48:35.687660 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.687635    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.689949 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.689930    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-ec2077-openshift-default-dockercfg-xl88m\""
Sep 18 06:48:35.696829 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.696802    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"]
Sep 18 06:48:35.700093 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.700069    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:35.794000 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.793956    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-data\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.794202 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.794053    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-envoy\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.794202 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.794102    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istiod-ca-cert\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.794202 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.794146    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cdkk\" (UniqueName: \"kubernetes.io/projected/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-kube-api-access-2cdkk\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.794403 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.794262    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-credential-socket\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.794403 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.794302    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-podinfo\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.794403 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.794344    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-token\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.794582 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.794416    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-workload-socket\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.794582 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.794486    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-workload-certs\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.895346 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.895309    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-data\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.895518 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.895370    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-envoy\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.895518 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.895398    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istiod-ca-cert\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.895518 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.895443    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cdkk\" (UniqueName: \"kubernetes.io/projected/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-kube-api-access-2cdkk\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.895518 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.895486    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-credential-socket\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.896160 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.895521    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-podinfo\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.896160 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.895554    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-token\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.896160 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.895629    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-workload-socket\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.896160 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.895673    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-workload-certs\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.896160 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.895844    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-data\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.896437 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.896211    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istiod-ca-cert\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.896437 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.896233    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-workload-certs\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.896437 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.896318    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-credential-socket\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.896437 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.896352    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-workload-socket\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.897715 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.897695    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-envoy\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.897951 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.897931    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-podinfo\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.902837 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.902816    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cdkk\" (UniqueName: \"kubernetes.io/projected/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-kube-api-access-2cdkk\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:35.902959 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:35.902877    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-token\") pod \"e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") " pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:36.001706 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:36.001632    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:36.039409 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:36.039369    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:36.278459 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:36.278427    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"]
Sep 18 06:48:36.280253 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:48:36.280210    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59f6e498_017b_4b6a_a0ab_5f5ce5b25adf.slice/crio-14c0b0aa5649f4c4703fd82a5582b78aef7b2cf76a5086afda0a968b7336371b WatchSource:0}: Error finding container 14c0b0aa5649f4c4703fd82a5582b78aef7b2cf76a5086afda0a968b7336371b: Status 404 returned error can't find the container with id 14c0b0aa5649f4c4703fd82a5582b78aef7b2cf76a5086afda0a968b7336371b
Sep 18 06:48:36.282721 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:36.282690    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:48:36.282804 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:36.282751    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:48:36.282804 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:36.282779    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:48:36.488444 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:36.488408    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:37.001055 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:37.001022    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb" event={"ID":"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf","Type":"ContainerStarted","Data":"2aacb02d2c8fe9153c5c42c4b858a2b1795192e5db01e612a204448bd85f87c3"}
Sep 18 06:48:37.001251 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:37.001074    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb" event={"ID":"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf","Type":"ContainerStarted","Data":"14c0b0aa5649f4c4703fd82a5582b78aef7b2cf76a5086afda0a968b7336371b"}
Sep 18 06:48:37.002383 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:37.002367    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:37.007402 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:37.007377    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:37.020813 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:37.020762    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb" podStartSLOduration=2.020748342 podStartE2EDuration="2.020748342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:48:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:48:37.018323378 +0000 UTC m=+1582.699851122" watchObservedRunningTime="2026-09-18 06:48:37.020748342 +0000 UTC m=+1582.702276086"
Sep 18 06:48:38.007286 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:38.007253    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:38.008695 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:38.008673    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:48:38.032874 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:38.032827    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:38.046909 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:38.046878    2568 patch_prober.go:28] interesting pod/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 06:48:38.047064 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:38.046950    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b" podUID="96c8cb03-28cf-41cb-829d-a7b9073bdb1c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 06:48:38.118961 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:38.118922    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:38.174533 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:38.174499    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:38.303003 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:38.302901    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:38.410624 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:38.410590    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:38.875613 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:38.875570    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:38.899494 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:38.899466    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"]
Sep 18 06:48:38.964676 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:38.964643    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:39.061704 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:39.061664    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:39.829243 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:39.829170    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:40.024393 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.024361    2568 generic.go:356] "Generic (PLEG): container finished" podID="96c8cb03-28cf-41cb-829d-a7b9073bdb1c" containerID="9e7bccb2d87e1176789e1b18af025644227fdf29474633d9914d34a16d3c13c3" exitCode=0
Sep 18 06:48:40.024569 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.024420    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b" event={"ID":"96c8cb03-28cf-41cb-829d-a7b9073bdb1c","Type":"ContainerDied","Data":"9e7bccb2d87e1176789e1b18af025644227fdf29474633d9914d34a16d3c13c3"}
Sep 18 06:48:40.024569 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.024506    2568 scope.go:117] "RemoveContainer" containerID="b3266bfbf38d05db250319aa0dd14a0a8ce457454a807ad0a57a8b316afca9e6"
Sep 18 06:48:40.146775 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.146751    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:48:40.243070 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.243036    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-workload-socket\") pod \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") "
Sep 18 06:48:40.243261 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.243172    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-data\") pod \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") "
Sep 18 06:48:40.243261 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.243198    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-credential-socket\") pod \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") "
Sep 18 06:48:40.243400 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.243266    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wn49c\" (UniqueName: \"kubernetes.io/projected/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-kube-api-access-wn49c\") pod \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") "
Sep 18 06:48:40.243400 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.243289    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-token\") pod \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") "
Sep 18 06:48:40.243400 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.243318    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-envoy\") pod \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") "
Sep 18 06:48:40.243400 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.243321    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "96c8cb03-28cf-41cb-829d-a7b9073bdb1c" (UID: "96c8cb03-28cf-41cb-829d-a7b9073bdb1c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:48:40.243400 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.243351    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-podinfo\") pod \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") "
Sep 18 06:48:40.243400 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.243383    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istiod-ca-cert\") pod \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") "
Sep 18 06:48:40.243704 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.243468    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-data" (OuterVolumeSpecName: "istio-data") pod "96c8cb03-28cf-41cb-829d-a7b9073bdb1c" (UID: "96c8cb03-28cf-41cb-829d-a7b9073bdb1c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:48:40.243704 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.243474    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-workload-certs\") pod \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\" (UID: \"96c8cb03-28cf-41cb-829d-a7b9073bdb1c\") "
Sep 18 06:48:40.243704 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.243564    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "96c8cb03-28cf-41cb-829d-a7b9073bdb1c" (UID: "96c8cb03-28cf-41cb-829d-a7b9073bdb1c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:48:40.243704 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.243684    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "96c8cb03-28cf-41cb-829d-a7b9073bdb1c" (UID: "96c8cb03-28cf-41cb-829d-a7b9073bdb1c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:48:40.243950 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.243928    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-workload-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:48:40.244015 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.243955    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-data\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:48:40.244015 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.243972    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-credential-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:48:40.244015 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.243985    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-workload-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:48:40.244163 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.244018    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "96c8cb03-28cf-41cb-829d-a7b9073bdb1c" (UID: "96c8cb03-28cf-41cb-829d-a7b9073bdb1c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 06:48:40.245614 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.245585    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-token" (OuterVolumeSpecName: "istio-token") pod "96c8cb03-28cf-41cb-829d-a7b9073bdb1c" (UID: "96c8cb03-28cf-41cb-829d-a7b9073bdb1c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:48:40.246035 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.246015    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-kube-api-access-wn49c" (OuterVolumeSpecName: "kube-api-access-wn49c") pod "96c8cb03-28cf-41cb-829d-a7b9073bdb1c" (UID: "96c8cb03-28cf-41cb-829d-a7b9073bdb1c"). InnerVolumeSpecName "kube-api-access-wn49c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:48:40.246108 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.246056    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "96c8cb03-28cf-41cb-829d-a7b9073bdb1c" (UID: "96c8cb03-28cf-41cb-829d-a7b9073bdb1c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:48:40.246190 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.246173    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "96c8cb03-28cf-41cb-829d-a7b9073bdb1c" (UID: "96c8cb03-28cf-41cb-829d-a7b9073bdb1c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 06:48:40.344847 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.344813    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wn49c\" (UniqueName: \"kubernetes.io/projected/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-kube-api-access-wn49c\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:48:40.344847 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.344843    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-token\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:48:40.345049 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.344856    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-envoy\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:48:40.345049 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.344868    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istio-podinfo\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:48:40.345049 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.344879    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96c8cb03-28cf-41cb-829d-a7b9073bdb1c-istiod-ca-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:48:40.456470 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:40.456439    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:41.032245 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:41.032188    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"
Sep 18 06:48:41.032438 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:41.032193    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b" event={"ID":"96c8cb03-28cf-41cb-829d-a7b9073bdb1c","Type":"ContainerDied","Data":"ef54da09fc4c67dda0a827831f2e136374359741de1db6995aba2f155b085ce7"}
Sep 18 06:48:41.032438 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:41.032388    2568 scope.go:117] "RemoveContainer" containerID="9e7bccb2d87e1176789e1b18af025644227fdf29474633d9914d34a16d3c13c3"
Sep 18 06:48:41.053757 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:41.053732    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"]
Sep 18 06:48:41.057511 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:41.057489    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b"]
Sep 18 06:48:41.069032 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:41.069006    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:41.829875 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:41.829833    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:42.448078 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:42.448043    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:42.544728 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:42.544696    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96c8cb03-28cf-41cb-829d-a7b9073bdb1c" path="/var/lib/kubelet/pods/96c8cb03-28cf-41cb-829d-a7b9073bdb1c/volumes"
Sep 18 06:48:42.547943 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:42.547922    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:42.648571 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:42.648530    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:42.897598 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:42.897574    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:42.988387 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:42.988358    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:43.103111 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:43.103080    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:43.103373 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:43.103150    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:43.118755 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:43.118726    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:43.433406 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:43.433372    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:44.061849 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:44.061822    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:48:44.485881 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:44.485851    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:48:47.526261 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:47.526199    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:47.574083 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:47.574052    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:48:53.918706 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:53.918659    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd" podUID="0059a78e-a837-4c30-a219-cdffed4da546" containerName="main" containerID="cri-o://9bdcf2907c6dbc4661e0ec993f5fb23b7afb2d206920d97d35196026d2c1e10e" gracePeriod=45
Sep 18 06:48:54.109358 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:54.109316    2568 generic.go:356] "Generic (PLEG): container finished" podID="0059a78e-a837-4c30-a219-cdffed4da546" containerID="9bdcf2907c6dbc4661e0ec993f5fb23b7afb2d206920d97d35196026d2c1e10e" exitCode=0
Sep 18 06:48:54.109501 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:54.109365    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd" event={"ID":"0059a78e-a837-4c30-a219-cdffed4da546","Type":"ContainerDied","Data":"9bdcf2907c6dbc4661e0ec993f5fb23b7afb2d206920d97d35196026d2c1e10e"}
Sep 18 06:48:54.173072 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:54.173004    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:48:54.288037 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:54.288007    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-home\") pod \"0059a78e-a837-4c30-a219-cdffed4da546\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") "
Sep 18 06:48:54.288180 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:54.288046    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0059a78e-a837-4c30-a219-cdffed4da546-tls-certs\") pod \"0059a78e-a837-4c30-a219-cdffed4da546\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") "
Sep 18 06:48:54.288180 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:54.288099    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-tmp-dir\") pod \"0059a78e-a837-4c30-a219-cdffed4da546\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") "
Sep 18 06:48:54.288180 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:54.288163    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-dshm\") pod \"0059a78e-a837-4c30-a219-cdffed4da546\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") "
Sep 18 06:48:54.288344 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:54.288197    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-model-cache\") pod \"0059a78e-a837-4c30-a219-cdffed4da546\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") "
Sep 18 06:48:54.288344 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:54.288213    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jtgg\" (UniqueName: \"kubernetes.io/projected/0059a78e-a837-4c30-a219-cdffed4da546-kube-api-access-6jtgg\") pod \"0059a78e-a837-4c30-a219-cdffed4da546\" (UID: \"0059a78e-a837-4c30-a219-cdffed4da546\") "
Sep 18 06:48:54.288344 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:54.288270    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-home" (OuterVolumeSpecName: "home") pod "0059a78e-a837-4c30-a219-cdffed4da546" (UID: "0059a78e-a837-4c30-a219-cdffed4da546"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:48:54.288516 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:54.288475    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0059a78e-a837-4c30-a219-cdffed4da546" (UID: "0059a78e-a837-4c30-a219-cdffed4da546"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:48:54.288586 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:54.288511    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-model-cache" (OuterVolumeSpecName: "model-cache") pod "0059a78e-a837-4c30-a219-cdffed4da546" (UID: "0059a78e-a837-4c30-a219-cdffed4da546"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:48:54.288652 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:54.288635    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:48:54.288703 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:54.288658    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:48:54.288703 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:54.288672    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:48:54.290326 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:54.290301    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-dshm" (OuterVolumeSpecName: "dshm") pod "0059a78e-a837-4c30-a219-cdffed4da546" (UID: "0059a78e-a837-4c30-a219-cdffed4da546"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:48:54.290438 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:54.290350    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0059a78e-a837-4c30-a219-cdffed4da546-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0059a78e-a837-4c30-a219-cdffed4da546" (UID: "0059a78e-a837-4c30-a219-cdffed4da546"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:48:54.290481 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:54.290438    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0059a78e-a837-4c30-a219-cdffed4da546-kube-api-access-6jtgg" (OuterVolumeSpecName: "kube-api-access-6jtgg") pod "0059a78e-a837-4c30-a219-cdffed4da546" (UID: "0059a78e-a837-4c30-a219-cdffed4da546"). InnerVolumeSpecName "kube-api-access-6jtgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:48:54.389303 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:54.389273    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jtgg\" (UniqueName: \"kubernetes.io/projected/0059a78e-a837-4c30-a219-cdffed4da546-kube-api-access-6jtgg\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:48:54.389303 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:54.389299    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0059a78e-a837-4c30-a219-cdffed4da546-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:48:54.389303 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:54.389310    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0059a78e-a837-4c30-a219-cdffed4da546-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:48:55.116179 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:55.116154    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"
Sep 18 06:48:55.116784 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:55.116193    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd" event={"ID":"0059a78e-a837-4c30-a219-cdffed4da546","Type":"ContainerDied","Data":"7ed4cca949c755df70d93e66616ead0741b7f07a7bbf6c846394cb7be485a9db"}
Sep 18 06:48:55.116784 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:55.116263    2568 scope.go:117] "RemoveContainer" containerID="9bdcf2907c6dbc4661e0ec993f5fb23b7afb2d206920d97d35196026d2c1e10e"
Sep 18 06:48:55.133089 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:55.133039    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"]
Sep 18 06:48:55.136414 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:55.136388    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-654463/e2e-conflict-a-24f27817-kserve-c6d8856c4-4tjdd"]
Sep 18 06:48:56.543850 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:48:56.543818    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0059a78e-a837-4c30-a219-cdffed4da546" path="/var/lib/kubelet/pods/0059a78e-a837-4c30-a219-cdffed4da546/volumes"
Sep 18 06:49:04.708061 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:04.707979    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:05.442491 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:05.442455    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:06.096641 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:06.096609    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:06.725048 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:06.725009    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:07.449456 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:07.449420    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:08.096179 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:08.096129    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:08.727076 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:08.727037    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:08.859234 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:08.859181    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:09.050414 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.050325    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 06:49:09.050414 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.050387    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb" podUID="59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 06:49:09.160005 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.159964    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:09.429246 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.429189    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"]
Sep 18 06:49:09.429914 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.429893    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96c8cb03-28cf-41cb-829d-a7b9073bdb1c" containerName="istio-proxy"
Sep 18 06:49:09.430096 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.430081    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="96c8cb03-28cf-41cb-829d-a7b9073bdb1c" containerName="istio-proxy"
Sep 18 06:49:09.430231 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.430205    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0059a78e-a837-4c30-a219-cdffed4da546" containerName="main"
Sep 18 06:49:09.430333 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.430322    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0059a78e-a837-4c30-a219-cdffed4da546" containerName="main"
Sep 18 06:49:09.430436 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.430427    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96c8cb03-28cf-41cb-829d-a7b9073bdb1c" containerName="istio-proxy"
Sep 18 06:49:09.430518 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.430508    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="96c8cb03-28cf-41cb-829d-a7b9073bdb1c" containerName="istio-proxy"
Sep 18 06:49:09.430717 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.430706    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0059a78e-a837-4c30-a219-cdffed4da546" containerName="main"
Sep 18 06:49:09.430805 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.430796    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="96c8cb03-28cf-41cb-829d-a7b9073bdb1c" containerName="istio-proxy"
Sep 18 06:49:09.430884 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.430875    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="96c8cb03-28cf-41cb-829d-a7b9073bdb1c" containerName="istio-proxy"
Sep 18 06:49:09.434756 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.434737    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.437778 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.437758    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-8fdd5b-openshift-default-dockercfg-v8656\""
Sep 18 06:49:09.446468 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.446444    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"]
Sep 18 06:49:09.456061 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.456035    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:09.541761 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.541731    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-podinfo\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.542040 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.542011    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-data\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.542164 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.542073    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm9qr\" (UniqueName: \"kubernetes.io/projected/01a52fd4-e07e-407c-9192-1467b42b6d78-kube-api-access-lm9qr\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.542164 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.542105    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-token\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.542164 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.542148    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a52fd4-e07e-407c-9192-1467b42b6d78-istiod-ca-cert\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.542372 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.542182    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-credential-socket\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.542372 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.542255    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-envoy\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.542372 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.542292    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-workload-socket\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.542372 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.542339    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-workload-certs\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.643190 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.643155    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-workload-certs\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.643456 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.643250    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-podinfo\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.643456 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.643316    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-data\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.643456 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.643358    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm9qr\" (UniqueName: \"kubernetes.io/projected/01a52fd4-e07e-407c-9192-1467b42b6d78-kube-api-access-lm9qr\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.643456 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.643388    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-token\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.643456 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.643457    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a52fd4-e07e-407c-9192-1467b42b6d78-istiod-ca-cert\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.643772 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.643492    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-credential-socket\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.643772 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.643533    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-envoy\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.643772 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.643558    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-workload-socket\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.643772 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.643734    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-workload-certs\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.644404 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.644107    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-data\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.644404 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.644358    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-credential-socket\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.644644 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.644619    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-workload-socket\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.644992 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.644968    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a52fd4-e07e-407c-9192-1467b42b6d78-istiod-ca-cert\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.646384 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.646361    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-envoy\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.646839 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.646819    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-podinfo\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.651950 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.651922    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-token\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.652896 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.652876    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm9qr\" (UniqueName: \"kubernetes.io/projected/01a52fd4-e07e-407c-9192-1467b42b6d78-kube-api-access-lm9qr\") pod \"e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") " pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.750297 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.750203    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:09.757968 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:09.757944    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:10.028562 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:10.028509    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"]
Sep 18 06:49:10.030023 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:49:10.029975    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01a52fd4_e07e_407c_9192_1467b42b6d78.slice/crio-453fd65e94977de399a86903dcd62359858e4c6f99d223776dba7b1f9b7c88b8 WatchSource:0}: Error finding container 453fd65e94977de399a86903dcd62359858e4c6f99d223776dba7b1f9b7c88b8: Status 404 returned error can't find the container with id 453fd65e94977de399a86903dcd62359858e4c6f99d223776dba7b1f9b7c88b8
Sep 18 06:49:10.033072 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:10.033033    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:49:10.033190 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:10.033122    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:49:10.033190 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:10.033163    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:49:10.211546 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:10.211467    2568 generic.go:356] "Generic (PLEG): container finished" podID="59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" containerID="2aacb02d2c8fe9153c5c42c4b858a2b1795192e5db01e612a204448bd85f87c3" exitCode=0
Sep 18 06:49:10.211708 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:10.211548    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb" event={"ID":"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf","Type":"ContainerDied","Data":"2aacb02d2c8fe9153c5c42c4b858a2b1795192e5db01e612a204448bd85f87c3"}
Sep 18 06:49:10.212091 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:10.212067    2568 scope.go:117] "RemoveContainer" containerID="2aacb02d2c8fe9153c5c42c4b858a2b1795192e5db01e612a204448bd85f87c3"
Sep 18 06:49:10.213488 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:10.213456    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:49:10.213608 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:10.213544    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:49:10.213608 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:10.213586    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:49:10.213696 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:10.213642    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s" event={"ID":"01a52fd4-e07e-407c-9192-1467b42b6d78","Type":"ContainerStarted","Data":"3b7e66f7a963c226cbbbea081d42efee06c033634da85dc1f654db247ab5280b"}
Sep 18 06:49:10.213696 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:10.213679    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s" event={"ID":"01a52fd4-e07e-407c-9192-1467b42b6d78","Type":"ContainerStarted","Data":"453fd65e94977de399a86903dcd62359858e4c6f99d223776dba7b1f9b7c88b8"}
Sep 18 06:49:10.251993 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:10.251940    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s" podStartSLOduration=1.251918101 podStartE2EDuration="1.251918101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:49:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:49:10.248064506 +0000 UTC m=+1615.929592254" watchObservedRunningTime="2026-09-18 06:49:10.251918101 +0000 UTC m=+1615.933445841"
Sep 18 06:49:10.750831 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:10.750800    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:11.221026 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:11.220990    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb" event={"ID":"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf","Type":"ContainerStarted","Data":"2f572786b3426fcadefc4bb9fb64f1225903b3a6d8aecb977e79affb2f7ff307"}
Sep 18 06:49:11.559206 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:11.559101    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:11.757783 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:11.757709    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:11.762282 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:11.762254    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:12.002570 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:12.002538    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:49:12.002780 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:12.002763    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 18 06:49:12.002830 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:12.002805    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb" podUID="59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 18 06:49:12.097367 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:12.097339    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:12.193431 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:12.193400    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:12.226945 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:12.226922    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:12.227847 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:12.227828    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:12.629587 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:12.629545    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:12.997862 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:12.997824    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:49:13.002185 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:13.002156    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 18 06:49:13.002346 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:13.002204    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb" podUID="59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 18 06:49:13.302579 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:13.302491    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:14.002576 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:14.002541    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 18 06:49:14.003017 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:14.002594    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb" podUID="59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 18 06:49:14.003510 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:14.003485    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:14.839416 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:14.839374    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:15.002099 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:15.002067    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 18 06:49:15.002281 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:15.002118    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb" podUID="59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 18 06:49:15.556641 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:15.556603    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:16.002080 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:16.002044    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 18 06:49:16.002278 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:16.002108    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb" podUID="59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 18 06:49:16.212166 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:16.212123    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:16.312928 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:16.312897    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:16.414999 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:16.414953    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:16.667987 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:16.667943    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:16.757937 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:16.757905    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:17.002452 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:17.002362    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 18 06:49:17.002452 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:17.002419    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb" podUID="59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 18 06:49:18.005616 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:18.005585    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:49:18.005991 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:18.005858    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:49:18.006484 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:18.006465    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 06:49:20.081386 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:20.081350    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:26.688667 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.688631    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"]
Sep 18 06:49:26.691912 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.691895    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:26.694360 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.694343    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-946609\"/\"distinct-2-w0-946609-kserve-self-signed-certs\""
Sep 18 06:49:26.702473 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.702447    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"]
Sep 18 06:49:26.740240 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.729385    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-home\") pod \"distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") " pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:26.740240 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.729502    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-tmp-dir\") pod \"distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") " pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:26.740240 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.729532    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-model-cache\") pod \"distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") " pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:26.740240 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.729598    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-dshm\") pod \"distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") " pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:26.740240 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.729670    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-tls-certs\") pod \"distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") " pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:26.740240 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.729715    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9mqb\" (UniqueName: \"kubernetes.io/projected/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-kube-api-access-w9mqb\") pod \"distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") " pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:26.830876 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.830835    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-tmp-dir\") pod \"distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") " pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:26.831054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.830890    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-model-cache\") pod \"distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") " pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:26.831054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.830960    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-dshm\") pod \"distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") " pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:26.831054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.831032    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-tls-certs\") pod \"distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") " pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:26.831292 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.831077    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9mqb\" (UniqueName: \"kubernetes.io/projected/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-kube-api-access-w9mqb\") pod \"distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") " pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:26.831292 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.831114    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-home\") pod \"distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") " pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:26.831639 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.831612    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-home\") pod \"distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") " pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:26.831899 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.831872    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-tmp-dir\") pod \"distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") " pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:26.832147 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.832123    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-model-cache\") pod \"distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") " pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:26.837243 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.836790    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-tls-certs\") pod \"distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") " pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:26.841175 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.838779    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-dshm\") pod \"distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") " pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:26.849010 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.848979    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9mqb\" (UniqueName: \"kubernetes.io/projected/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-kube-api-access-w9mqb\") pod \"distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") " pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:26.881170 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:26.881116    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:27.002160 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:27.002076    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:27.301814 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:27.301787    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"]
Sep 18 06:49:27.303332 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:49:27.303303    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a9ba24a_6256_4969_a7f2_d39d8d95cfc1.slice/crio-b14f202e10c5b58052321d362c02f76094866995ce8d773ecbe777ad27c21fbd WatchSource:0}: Error finding container b14f202e10c5b58052321d362c02f76094866995ce8d773ecbe777ad27c21fbd: Status 404 returned error can't find the container with id b14f202e10c5b58052321d362c02f76094866995ce8d773ecbe777ad27c21fbd
Sep 18 06:49:27.328836 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:27.328814    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj" event={"ID":"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1","Type":"ContainerStarted","Data":"b14f202e10c5b58052321d362c02f76094866995ce8d773ecbe777ad27c21fbd"}
Sep 18 06:49:28.343772 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:28.343726    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj" event={"ID":"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1","Type":"ContainerStarted","Data":"bf96acc107bce0990422e8eaec71591832f5ab9e7b7fa3c735f62b97abee911f"}
Sep 18 06:49:28.361490 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:28.361429    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj" podStartSLOduration=2.361408663 podStartE2EDuration="2.361408663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:49:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:49:28.359052871 +0000 UTC m=+1634.040580615" watchObservedRunningTime="2026-09-18 06:49:28.361408663 +0000 UTC m=+1634.042936408"
Sep 18 06:49:34.383565 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:34.383532    2568 generic.go:356] "Generic (PLEG): container finished" podID="01a52fd4-e07e-407c-9192-1467b42b6d78" containerID="3b7e66f7a963c226cbbbea081d42efee06c033634da85dc1f654db247ab5280b" exitCode=0
Sep 18 06:49:34.383963 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:34.383605    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s" event={"ID":"01a52fd4-e07e-407c-9192-1467b42b6d78","Type":"ContainerDied","Data":"3b7e66f7a963c226cbbbea081d42efee06c033634da85dc1f654db247ab5280b"}
Sep 18 06:49:34.384077 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:34.384063    2568 scope.go:117] "RemoveContainer" containerID="3b7e66f7a963c226cbbbea081d42efee06c033634da85dc1f654db247ab5280b"
Sep 18 06:49:34.385209 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:34.385184    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:49:34.385286 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:34.385272    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:49:34.385337 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:34.385305    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:49:35.394207 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:35.394169    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s" event={"ID":"01a52fd4-e07e-407c-9192-1467b42b6d78","Type":"ContainerStarted","Data":"dcf16968054977490b4f62edd85464b3823a35b8087bdf1d21cddea3a0fbf0f8"}
Sep 18 06:49:35.751611 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:35.751577    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:35.752015 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:35.751627    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 18 06:49:35.752206 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:35.752180    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s" podUID="01a52fd4-e07e-407c-9192-1467b42b6d78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 18 06:49:36.751533 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:36.751487    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 18 06:49:36.752036 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:36.751565    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s" podUID="01a52fd4-e07e-407c-9192-1467b42b6d78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 18 06:49:37.002858 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:37.002771    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:37.003029 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:37.002965    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:37.015273 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:37.015247    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:37.415933 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:37.415904    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:49:37.751736 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:37.751638    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 18 06:49:37.752324 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:37.752288    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s" podUID="01a52fd4-e07e-407c-9192-1467b42b6d78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 18 06:49:38.430712 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:38.430670    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:38.751278 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:38.751163    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 18 06:49:38.751553 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:38.751522    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s" podUID="01a52fd4-e07e-407c-9192-1467b42b6d78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 18 06:49:39.051748 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:39.051653    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:39.666999 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:39.666966    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:39.751288 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:39.751250    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 18 06:49:39.751495 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:39.751320    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s" podUID="01a52fd4-e07e-407c-9192-1467b42b6d78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 18 06:49:40.378296 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:40.378258    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:40.754975 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:40.754946    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:40.755262 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:40.755241    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:40.755850 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:40.755831    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 06:49:41.015284 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:41.015173    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:41.831565 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:41.831532    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:42.126898 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:42.126809    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:42.444369 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:42.444335    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:43.082399 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:43.082352    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:49:45.685447 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:49:45.685413    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:50:08.724568 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:08.724521    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:15.695143 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:15.695111    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:50:20.265479 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:20.265441    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:20.857833 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:20.857801    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:21.621389 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:21.621351    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:22.227513 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:22.227483    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:22.857206 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:22.857175    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:23.731830 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:23.731789    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:24.340600 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:24.340558    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:25.006407 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:25.006367    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:25.640307 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:25.640273    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:35.494492 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:35.494414    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:35.593986 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:35.593956    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:35.672629 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:35.672601    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:35.722254 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:35.722207    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:36.208082 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:36.208051    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:36.917240 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:36.917189    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:37.627819 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:37.627782    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:38.231848 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:38.231807    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:38.601187 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:38.601098    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:38.853069 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:38.852979    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:39.638998 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:39.638962    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:40.335624 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:40.335592    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:43.593289 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:43.593238    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"]
Sep 18 06:50:43.638366 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:43.638333    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:43.692195 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:43.692160    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:45.322970 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:45.322933    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:45.370950 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:45.370920    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:55.697744 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:55.697709    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:50:57.044321 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:57.044287    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:57.802990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:57.802951    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:50:58.613011 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:58.612971    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b" podUID="f9699e03-05ac-4b5a-9555-52a5d39b62e3" containerName="main" containerID="cri-o://0ca6566058575d9d56fb741cc503f0cfe605aea078e1cbd385cff42df0dce63a" gracePeriod=45
Sep 18 06:50:58.862146 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:58.862118    2568 generic.go:356] "Generic (PLEG): container finished" podID="f9699e03-05ac-4b5a-9555-52a5d39b62e3" containerID="0ca6566058575d9d56fb741cc503f0cfe605aea078e1cbd385cff42df0dce63a" exitCode=0
Sep 18 06:50:58.862301 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:58.862173    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b" event={"ID":"f9699e03-05ac-4b5a-9555-52a5d39b62e3","Type":"ContainerDied","Data":"0ca6566058575d9d56fb741cc503f0cfe605aea078e1cbd385cff42df0dce63a"}
Sep 18 06:50:58.862301 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:58.862249    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b" event={"ID":"f9699e03-05ac-4b5a-9555-52a5d39b62e3","Type":"ContainerDied","Data":"f5772dea7da20b2cfefe5ab5b6984d448a83ec2e719ad80a79825fec6514c216"}
Sep 18 06:50:58.862301 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:58.862269    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f5772dea7da20b2cfefe5ab5b6984d448a83ec2e719ad80a79825fec6514c216"
Sep 18 06:50:58.868853 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:58.868802    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:50:58.931987 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:58.931953    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xqfr\" (UniqueName: \"kubernetes.io/projected/f9699e03-05ac-4b5a-9555-52a5d39b62e3-kube-api-access-9xqfr\") pod \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") "
Sep 18 06:50:58.932162 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:58.932038    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-model-cache\") pod \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") "
Sep 18 06:50:58.932162 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:58.932068    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-tmp-dir\") pod \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") "
Sep 18 06:50:58.932162 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:58.932116    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-dshm\") pod \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") "
Sep 18 06:50:58.932350 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:58.932199    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-home\") pod \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") "
Sep 18 06:50:58.932350 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:58.932320    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9699e03-05ac-4b5a-9555-52a5d39b62e3-tls-certs\") pod \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\" (UID: \"f9699e03-05ac-4b5a-9555-52a5d39b62e3\") "
Sep 18 06:50:58.932738 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:58.932679    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-home" (OuterVolumeSpecName: "home") pod "f9699e03-05ac-4b5a-9555-52a5d39b62e3" (UID: "f9699e03-05ac-4b5a-9555-52a5d39b62e3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:50:58.932738 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:58.932693    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f9699e03-05ac-4b5a-9555-52a5d39b62e3" (UID: "f9699e03-05ac-4b5a-9555-52a5d39b62e3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:50:58.932918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:58.932892    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-model-cache" (OuterVolumeSpecName: "model-cache") pod "f9699e03-05ac-4b5a-9555-52a5d39b62e3" (UID: "f9699e03-05ac-4b5a-9555-52a5d39b62e3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:50:58.934689 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:58.934662    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9699e03-05ac-4b5a-9555-52a5d39b62e3-kube-api-access-9xqfr" (OuterVolumeSpecName: "kube-api-access-9xqfr") pod "f9699e03-05ac-4b5a-9555-52a5d39b62e3" (UID: "f9699e03-05ac-4b5a-9555-52a5d39b62e3"). InnerVolumeSpecName "kube-api-access-9xqfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:50:58.934906 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:58.934879    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9699e03-05ac-4b5a-9555-52a5d39b62e3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f9699e03-05ac-4b5a-9555-52a5d39b62e3" (UID: "f9699e03-05ac-4b5a-9555-52a5d39b62e3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:50:58.934996 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:58.934925    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-dshm" (OuterVolumeSpecName: "dshm") pod "f9699e03-05ac-4b5a-9555-52a5d39b62e3" (UID: "f9699e03-05ac-4b5a-9555-52a5d39b62e3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:50:59.034213 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:59.034172    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9699e03-05ac-4b5a-9555-52a5d39b62e3-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:50:59.034213 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:59.034208    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xqfr\" (UniqueName: \"kubernetes.io/projected/f9699e03-05ac-4b5a-9555-52a5d39b62e3-kube-api-access-9xqfr\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:50:59.034470 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:59.034254    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:50:59.034470 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:59.034267    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:50:59.034470 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:59.034280    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:50:59.034470 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:59.034291    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9699e03-05ac-4b5a-9555-52a5d39b62e3-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:50:59.867284 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:59.867256    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"
Sep 18 06:50:59.894504 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:59.894477    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"]
Sep 18 06:50:59.899531 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:50:59.899508    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-93683709/e2e-lifecycle-model-93683709-kserve-7f88d679dc-77n9b"]
Sep 18 06:51:00.544154 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:00.544107    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9699e03-05ac-4b5a-9555-52a5d39b62e3" path="/var/lib/kubelet/pods/f9699e03-05ac-4b5a-9555-52a5d39b62e3/volumes"
Sep 18 06:51:09.270995 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:09.270961    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:09.371475 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:09.371444    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:16.688014 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:16.687978    2568 scope.go:117] "RemoveContainer" containerID="dedbd10e8110ab29f8475ce52e30144a6babc53380b8322796542faaaf769f76"
Sep 18 06:51:18.386620 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.386583    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"]
Sep 18 06:51:18.387050 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.387038    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9699e03-05ac-4b5a-9555-52a5d39b62e3" containerName="main"
Sep 18 06:51:18.387094 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.387051    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9699e03-05ac-4b5a-9555-52a5d39b62e3" containerName="main"
Sep 18 06:51:18.387199 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.387166    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9699e03-05ac-4b5a-9555-52a5d39b62e3" containerName="main"
Sep 18 06:51:18.390244 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.390206    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:18.392667 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.392651    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-654463\"/\"distinct-2-w4-654463-kserve-self-signed-certs\""
Sep 18 06:51:18.400038 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.400007    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"]
Sep 18 06:51:18.512533 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.512499    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:18.541323 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.541294    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-model-cache\") pod \"distinct-2-w4-654463-kserve-55986bd45d-2sq48\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") " pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:18.541323 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.541330    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72047399-e64a-4137-b4dd-37b8bfcc32c1-tls-certs\") pod \"distinct-2-w4-654463-kserve-55986bd45d-2sq48\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") " pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:18.541580 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.541421    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-tmp-dir\") pod \"distinct-2-w4-654463-kserve-55986bd45d-2sq48\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") " pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:18.541580 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.541506    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-dshm\") pod \"distinct-2-w4-654463-kserve-55986bd45d-2sq48\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") " pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:18.541580 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.541542    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f86f\" (UniqueName: \"kubernetes.io/projected/72047399-e64a-4137-b4dd-37b8bfcc32c1-kube-api-access-7f86f\") pod \"distinct-2-w4-654463-kserve-55986bd45d-2sq48\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") " pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:18.541739 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.541623    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-home\") pod \"distinct-2-w4-654463-kserve-55986bd45d-2sq48\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") " pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:18.643235 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.643145    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-tmp-dir\") pod \"distinct-2-w4-654463-kserve-55986bd45d-2sq48\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") " pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:18.643405 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.643277    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-dshm\") pod \"distinct-2-w4-654463-kserve-55986bd45d-2sq48\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") " pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:18.643405 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.643328    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f86f\" (UniqueName: \"kubernetes.io/projected/72047399-e64a-4137-b4dd-37b8bfcc32c1-kube-api-access-7f86f\") pod \"distinct-2-w4-654463-kserve-55986bd45d-2sq48\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") " pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:18.643405 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.643362    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-home\") pod \"distinct-2-w4-654463-kserve-55986bd45d-2sq48\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") " pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:18.643576 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.643427    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-model-cache\") pod \"distinct-2-w4-654463-kserve-55986bd45d-2sq48\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") " pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:18.643576 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.643448    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72047399-e64a-4137-b4dd-37b8bfcc32c1-tls-certs\") pod \"distinct-2-w4-654463-kserve-55986bd45d-2sq48\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") " pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:18.643576 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.643518    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-tmp-dir\") pod \"distinct-2-w4-654463-kserve-55986bd45d-2sq48\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") " pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:18.643759 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.643737    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-home\") pod \"distinct-2-w4-654463-kserve-55986bd45d-2sq48\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") " pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:18.643805 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.643774    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-model-cache\") pod \"distinct-2-w4-654463-kserve-55986bd45d-2sq48\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") " pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:18.645572 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.645548    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-dshm\") pod \"distinct-2-w4-654463-kserve-55986bd45d-2sq48\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") " pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:18.645839 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.645824    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72047399-e64a-4137-b4dd-37b8bfcc32c1-tls-certs\") pod \"distinct-2-w4-654463-kserve-55986bd45d-2sq48\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") " pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:18.650980 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.650964    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f86f\" (UniqueName: \"kubernetes.io/projected/72047399-e64a-4137-b4dd-37b8bfcc32c1-kube-api-access-7f86f\") pod \"distinct-2-w4-654463-kserve-55986bd45d-2sq48\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") " pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:18.699645 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.699623    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:18.969159 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.969120    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"]
Sep 18 06:51:18.970180 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:51:18.970143    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72047399_e64a_4137_b4dd_37b8bfcc32c1.slice/crio-0adde68dfc2820f0facd75195a1e2c535c51efad8a9803c32eb3a875374c54ca WatchSource:0}: Error finding container 0adde68dfc2820f0facd75195a1e2c535c51efad8a9803c32eb3a875374c54ca: Status 404 returned error can't find the container with id 0adde68dfc2820f0facd75195a1e2c535c51efad8a9803c32eb3a875374c54ca
Sep 18 06:51:18.979480 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:18.979458    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48" event={"ID":"72047399-e64a-4137-b4dd-37b8bfcc32c1","Type":"ContainerStarted","Data":"0adde68dfc2820f0facd75195a1e2c535c51efad8a9803c32eb3a875374c54ca"}
Sep 18 06:51:19.987046 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:19.987013    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48" event={"ID":"72047399-e64a-4137-b4dd-37b8bfcc32c1","Type":"ContainerStarted","Data":"92c467d08c6528434cbb1d4e62f086db8f4f8887dc856bf2c4f184271ba525b5"}
Sep 18 06:51:20.002374 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:20.002330    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48" podStartSLOduration=2.002274175 podStartE2EDuration="2.002274175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:51:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:51:20.002152399 +0000 UTC m=+1745.683680142" watchObservedRunningTime="2026-09-18 06:51:20.002274175 +0000 UTC m=+1745.683801918"
Sep 18 06:51:20.708647 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:20.708611    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:20.840329 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:20.840297    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:26.273566 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:26.273523    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:28.692725 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:28.692693    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:51:28.700119 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:28.700090    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:28.700306 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:28.700129    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:28.712567 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:28.712545    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:29.054764 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.054676    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:51:29.686904 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.686866    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"]
Sep 18 06:51:29.689699 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.689684    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:29.692295 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.692272    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-8fdd5b\"/\"default-dockercfg-nl8gc\""
Sep 18 06:51:29.693205 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.693187    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-8fdd5b\"/\"kube-root-ca.crt\""
Sep 18 06:51:29.693582 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.693274    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-8fdd5b\"/\"openshift-service-ca.crt\""
Sep 18 06:51:29.693582 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.693274    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-8fdd5b\"/\"auth-test-model-8fdd5b-kserve-self-signed-certs\""
Sep 18 06:51:29.698883 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.698860    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"]
Sep 18 06:51:29.712459 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.712435    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:29.759644 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.759618    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-model-cache\") pod \"auth-test-model-8fdd5b-kserve-658b55b847-jnnds\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:29.759831 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.759649    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5gth\" (UniqueName: \"kubernetes.io/projected/24eb3bee-e209-4fe4-a1df-095c77af875d-kube-api-access-s5gth\") pod \"auth-test-model-8fdd5b-kserve-658b55b847-jnnds\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:29.759831 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.759743    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-home\") pod \"auth-test-model-8fdd5b-kserve-658b55b847-jnnds\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:29.759831 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.759797    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-dshm\") pod \"auth-test-model-8fdd5b-kserve-658b55b847-jnnds\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:29.759997 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.759887    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-tmp-dir\") pod \"auth-test-model-8fdd5b-kserve-658b55b847-jnnds\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:29.759997 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.759911    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24eb3bee-e209-4fe4-a1df-095c77af875d-tls-certs\") pod \"auth-test-model-8fdd5b-kserve-658b55b847-jnnds\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:29.860860 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.860828    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-model-cache\") pod \"auth-test-model-8fdd5b-kserve-658b55b847-jnnds\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:29.860860 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.860862    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5gth\" (UniqueName: \"kubernetes.io/projected/24eb3bee-e209-4fe4-a1df-095c77af875d-kube-api-access-s5gth\") pod \"auth-test-model-8fdd5b-kserve-658b55b847-jnnds\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:29.861072 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.860881    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-home\") pod \"auth-test-model-8fdd5b-kserve-658b55b847-jnnds\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:29.861072 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.860895    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-dshm\") pod \"auth-test-model-8fdd5b-kserve-658b55b847-jnnds\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:29.861072 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.860930    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-tmp-dir\") pod \"auth-test-model-8fdd5b-kserve-658b55b847-jnnds\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:29.861072 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.860954    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24eb3bee-e209-4fe4-a1df-095c77af875d-tls-certs\") pod \"auth-test-model-8fdd5b-kserve-658b55b847-jnnds\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:29.861378 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.861355    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-home\") pod \"auth-test-model-8fdd5b-kserve-658b55b847-jnnds\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:29.861446 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.861377    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-model-cache\") pod \"auth-test-model-8fdd5b-kserve-658b55b847-jnnds\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:29.861446 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.861404    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-tmp-dir\") pod \"auth-test-model-8fdd5b-kserve-658b55b847-jnnds\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:29.863264 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.863241    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-dshm\") pod \"auth-test-model-8fdd5b-kserve-658b55b847-jnnds\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:29.863508 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.863491    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24eb3bee-e209-4fe4-a1df-095c77af875d-tls-certs\") pod \"auth-test-model-8fdd5b-kserve-658b55b847-jnnds\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:29.868109 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:29.868086    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5gth\" (UniqueName: \"kubernetes.io/projected/24eb3bee-e209-4fe4-a1df-095c77af875d-kube-api-access-s5gth\") pod \"auth-test-model-8fdd5b-kserve-658b55b847-jnnds\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:30.000503 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:30.000416    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:30.272081 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:30.272050    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"]
Sep 18 06:51:30.273255 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:51:30.273209    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24eb3bee_e209_4fe4_a1df_095c77af875d.slice/crio-4264faf2056ff498609715c0dbe9c2f612acfa5cfb2edd6dfa8600076665f552 WatchSource:0}: Error finding container 4264faf2056ff498609715c0dbe9c2f612acfa5cfb2edd6dfa8600076665f552: Status 404 returned error can't find the container with id 4264faf2056ff498609715c0dbe9c2f612acfa5cfb2edd6dfa8600076665f552
Sep 18 06:51:31.065418 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:31.065376    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds" event={"ID":"24eb3bee-e209-4fe4-a1df-095c77af875d","Type":"ContainerStarted","Data":"71cec49caf841b30ee89cf83bcd367213628f4cfcac456356401841a0c83d32e"}
Sep 18 06:51:31.065418 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:31.065430    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds" event={"ID":"24eb3bee-e209-4fe4-a1df-095c77af875d","Type":"ContainerStarted","Data":"4264faf2056ff498609715c0dbe9c2f612acfa5cfb2edd6dfa8600076665f552"}
Sep 18 06:51:31.083026 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:31.082976    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds" podStartSLOduration=2.082962412 podStartE2EDuration="2.082962412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:51:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:51:31.080645064 +0000 UTC m=+1756.762172881" watchObservedRunningTime="2026-09-18 06:51:31.082962412 +0000 UTC m=+1756.764490155"
Sep 18 06:51:34.840103 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:34.840062    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:37.613727 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:37.613677    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:40.001545 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:40.001510    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:40.001934 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:40.001786    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:40.014968 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:40.014946    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:40.127913 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:40.127884    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:51:40.636160 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:40.636116    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:40.692070 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:40.692041    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:40.872797 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:40.872765    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:42.053918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:42.053879    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:42.153696 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:42.153662    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:42.253917 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:42.253885    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:42.818813 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:42.818777    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:42.918738 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:42.918705    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:43.018729 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:43.018701    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:44.177657 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:44.177624    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:46.165598 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:46.165561    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:49.951162 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:49.951104    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:50.009498 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:50.009460    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:50.572512 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:50.572477    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:56.458769 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:56.458732    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:56.565939 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:56.565898    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:51:56.658896 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:51:56.658848    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:52:07.098949 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:07.098921    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:52:07.316233 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:07.316185    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:52:13.835141 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:13.835103    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:52:14.783745 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:14.783705    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:52:28.191726 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:28.191675    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:52:33.868693 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:33.868646    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:52:34.216294 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:34.216257    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:52:34.272381 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:34.272337    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:52:34.478358 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:34.478278    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:52:34.602686 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:34.602647    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:52:36.438545 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:36.438509    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:52:38.559406 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:38.559367    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:52:38.844578 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:38.844484    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:52:39.571199 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:39.571145    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:52:40.429718 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:40.429675    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:52:41.137953 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:41.137916    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:52:42.037521 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:42.037490    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:52:42.138641 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:42.138610    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:52:42.238355 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:42.238322    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:52:42.480801 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:42.480771    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:52:42.576598 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:42.576552    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:52:52.092518 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:52.092486    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:52:52.984373 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:52.984336    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"]
Sep 18 06:52:52.988145 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:52.988123    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:52:52.990663 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:52.990639    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-946609\"/\"embedding-w0-946609-kserve-self-signed-certs\""
Sep 18 06:52:52.996751 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:52.996725    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"]
Sep 18 06:52:53.024935 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:53.024891    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:52:53.150799 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:53.150764    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-home\") pod \"embedding-w0-946609-kserve-748855cffd-fnjp6\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") " pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:52:53.150799 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:53.150799    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-model-cache\") pod \"embedding-w0-946609-kserve-748855cffd-fnjp6\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") " pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:52:53.151307 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:53.150818    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zn4x\" (UniqueName: \"kubernetes.io/projected/1f9f4452-bbeb-4f35-880e-45ca235691b9-kube-api-access-2zn4x\") pod \"embedding-w0-946609-kserve-748855cffd-fnjp6\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") " pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:52:53.151307 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:53.150875    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-tmp-dir\") pod \"embedding-w0-946609-kserve-748855cffd-fnjp6\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") " pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:52:53.151307 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:53.151002    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f9f4452-bbeb-4f35-880e-45ca235691b9-tls-certs\") pod \"embedding-w0-946609-kserve-748855cffd-fnjp6\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") " pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:52:53.151307 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:53.151033    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-dshm\") pod \"embedding-w0-946609-kserve-748855cffd-fnjp6\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") " pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:52:53.251883 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:53.251777    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-model-cache\") pod \"embedding-w0-946609-kserve-748855cffd-fnjp6\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") " pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:52:53.251883 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:53.251819    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zn4x\" (UniqueName: \"kubernetes.io/projected/1f9f4452-bbeb-4f35-880e-45ca235691b9-kube-api-access-2zn4x\") pod \"embedding-w0-946609-kserve-748855cffd-fnjp6\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") " pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:52:53.251883 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:53.251840    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-tmp-dir\") pod \"embedding-w0-946609-kserve-748855cffd-fnjp6\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") " pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:52:53.252238 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:53.251899    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f9f4452-bbeb-4f35-880e-45ca235691b9-tls-certs\") pod \"embedding-w0-946609-kserve-748855cffd-fnjp6\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") " pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:52:53.252238 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:53.251917    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-dshm\") pod \"embedding-w0-946609-kserve-748855cffd-fnjp6\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") " pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:52:53.252238 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:53.252069    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-home\") pod \"embedding-w0-946609-kserve-748855cffd-fnjp6\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") " pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:52:53.252437 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:53.252376    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-model-cache\") pod \"embedding-w0-946609-kserve-748855cffd-fnjp6\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") " pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:52:53.252437 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:53.252399    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-tmp-dir\") pod \"embedding-w0-946609-kserve-748855cffd-fnjp6\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") " pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:52:53.252567 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:53.252435    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-home\") pod \"embedding-w0-946609-kserve-748855cffd-fnjp6\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") " pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:52:53.254426 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:53.254403    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-dshm\") pod \"embedding-w0-946609-kserve-748855cffd-fnjp6\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") " pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:52:53.255117 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:53.255098    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f9f4452-bbeb-4f35-880e-45ca235691b9-tls-certs\") pod \"embedding-w0-946609-kserve-748855cffd-fnjp6\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") " pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:52:53.259807 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:53.259780    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zn4x\" (UniqueName: \"kubernetes.io/projected/1f9f4452-bbeb-4f35-880e-45ca235691b9-kube-api-access-2zn4x\") pod \"embedding-w0-946609-kserve-748855cffd-fnjp6\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") " pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:52:53.298210 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:53.298185    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:52:53.572351 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:53.572320    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"]
Sep 18 06:52:53.573494 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:52:53.573457    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f9f4452_bbeb_4f35_880e_45ca235691b9.slice/crio-a0c2d10253ec00a00919464b7f8197905f3da29dbadb8d2f66c57b5f69baf071 WatchSource:0}: Error finding container a0c2d10253ec00a00919464b7f8197905f3da29dbadb8d2f66c57b5f69baf071: Status 404 returned error can't find the container with id a0c2d10253ec00a00919464b7f8197905f3da29dbadb8d2f66c57b5f69baf071
Sep 18 06:52:54.561598 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:54.561567    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6" event={"ID":"1f9f4452-bbeb-4f35-880e-45ca235691b9","Type":"ContainerStarted","Data":"39b222d48642db10fdfe134c9afba27c799f19933b6bc3f11308586aca67bef1"}
Sep 18 06:52:54.561993 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:54.561607    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6" event={"ID":"1f9f4452-bbeb-4f35-880e-45ca235691b9","Type":"ContainerStarted","Data":"a0c2d10253ec00a00919464b7f8197905f3da29dbadb8d2f66c57b5f69baf071"}
Sep 18 06:52:54.578126 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:52:54.578079    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6" podStartSLOduration=2.578063396 podStartE2EDuration="2.578063396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:52:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:52:54.575689601 +0000 UTC m=+1840.257217345" watchObservedRunningTime="2026-09-18 06:52:54.578063396 +0000 UTC m=+1840.259591170"
Sep 18 06:53:03.299140 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:03.299092    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:53:03.299140 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:03.299155    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:53:03.312073 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:03.312040    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:53:03.631516 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:03.631435    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:53:04.334433 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:04.334399    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:04.942124 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:04.942094    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:05.562686 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:05.562657    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:06.326395 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:06.326365    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:06.947164 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:06.947130    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:07.174958 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:07.174921    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:07.608581 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:07.608542    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:07.708772 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:07.708744    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:07.897807 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:07.897780    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:08.103821 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:08.103787    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:08.531119 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:08.531073    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:09.175852 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:09.175820    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:10.731430 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:10.731392    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:11.360385 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:11.360346    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:11.452173 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:11.452134    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:11.554055 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:11.554025    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:12.350484 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:12.350452    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:13.069705 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:13.069672    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:13.804228 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:13.804178    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:14.454424 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:14.454390    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:15.096967 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:15.096927    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:15.747031 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:15.747001    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:16.373706 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:16.373672    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:16.476534 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:16.476438    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:16.710942 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:16.710910    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:16.826580 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:16.826548    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:16.923102 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:16.923071    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:38.865836 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:38.865758    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:39.468683 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:39.468643    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:40.144939 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:40.144909    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:40.759032 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:40.758999    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:41.361854 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:41.361825    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:41.530272 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:41.530204    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:42.169330 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:42.169295    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:42.762357 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:42.762323    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:43.539886 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:43.539850    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:44.151279 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:44.151213    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:44.212644 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:44.212613    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:53:44.911195 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:44.911160    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:45.745654 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:45.745623    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:46.422231 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:46.422197    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:47.125813 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:47.125764    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:47.773689 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:47.773647    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:48.707668 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:48.707634    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:49.340527 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:49.340495    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:50.130582 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:50.130540    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:50.827913 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:50.827879    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:54.338452 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:54.338394    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:59.878813 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:59.878772    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:53:59.938734 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:53:59.938699    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:00.001211 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:00.001182    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:00.065187 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:00.065151    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:00.447393 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:00.447363    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:00.573581 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:00.573279    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:01.234942 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:01.234902    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:02.020259 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:02.020206    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:02.643817 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:02.643788    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:03.248105 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:03.248073    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:04.029330 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:04.029296    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:04.636900 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:04.636861    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:05.248901 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:05.248866    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:05.951401 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:05.951365    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:06.816722 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:06.816690    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:06.920039 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:06.920009    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:07.064909 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:07.064872    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:07.858152 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:07.858114    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:08.539474 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:08.539395    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:09.314389 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:09.314343    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:09.804188 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:09.804143    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:10.111119 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:10.110991    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:10.830531 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:10.830498    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:11.538597 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:11.538568    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:12.350186 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:12.350149    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:13.050236 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:13.050184    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:13.902154 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:13.902117    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:14.533429 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:14.533371    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:14.788847 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:14.788760    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"]
Sep 18 06:54:14.804701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:14.804672    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"]
Sep 18 06:54:14.824027 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:14.823996    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"]
Sep 18 06:54:14.882587 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:14.882557    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:15.229819 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:15.229781    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:15.842005 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:15.841972    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:17.531590 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:17.531556    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:20.055759 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:20.055725    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:23.095336 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:23.095302    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:23.174758 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:23.174729    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:23.232754 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:23.232722    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:23.334536 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:23.334506    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:23.631118 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:23.631083    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:23.731478 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:23.731440    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:24.417546 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:24.417514    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:25.037954 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:25.037918    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:25.666886 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:25.666855    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:26.435508 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:26.435473    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:26.886879 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:26.886803    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:54:27.099525 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:27.099494    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:27.627525 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:27.627495    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"]
Sep 18 06:54:27.645772 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:27.645742    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-946609-29828565-dr9tz"]
Sep 18 06:54:27.648720 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:27.648695    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:27.649512 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:27.649490    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-946609-29828565-dr9tz"]
Sep 18 06:54:27.712584 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:27.712551    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:27.780206 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:27.780178    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:27.879341 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:27.879265    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:28.103243 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:28.103187    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f" podUID="ea698d13-b785-4a73-8563-04df09ce4424" containerName="maas-api" containerID="cri-o://60811aded78ad3264eebbee762843413185aaefbda244276ba68e80d0e013bb8" gracePeriod=30
Sep 18 06:54:28.157176 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:28.157147    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:28.362188 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:28.362165    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"
Sep 18 06:54:28.395458 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:28.395432    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea698d13-b785-4a73-8563-04df09ce4424-maas-api-tls\") pod \"ea698d13-b785-4a73-8563-04df09ce4424\" (UID: \"ea698d13-b785-4a73-8563-04df09ce4424\") "
Sep 18 06:54:28.395618 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:28.395478    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jdjh\" (UniqueName: \"kubernetes.io/projected/ea698d13-b785-4a73-8563-04df09ce4424-kube-api-access-7jdjh\") pod \"ea698d13-b785-4a73-8563-04df09ce4424\" (UID: \"ea698d13-b785-4a73-8563-04df09ce4424\") "
Sep 18 06:54:28.395618 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:28.395508    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea698d13-b785-4a73-8563-04df09ce4424-metrics-certs\") pod \"ea698d13-b785-4a73-8563-04df09ce4424\" (UID: \"ea698d13-b785-4a73-8563-04df09ce4424\") "
Sep 18 06:54:28.395618 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:28.395532    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea698d13-b785-4a73-8563-04df09ce4424-openshift-service-ca\") pod \"ea698d13-b785-4a73-8563-04df09ce4424\" (UID: \"ea698d13-b785-4a73-8563-04df09ce4424\") "
Sep 18 06:54:28.396063 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:28.396019    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea698d13-b785-4a73-8563-04df09ce4424-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ea698d13-b785-4a73-8563-04df09ce4424" (UID: "ea698d13-b785-4a73-8563-04df09ce4424"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 06:54:28.397575 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:28.397552    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea698d13-b785-4a73-8563-04df09ce4424-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ea698d13-b785-4a73-8563-04df09ce4424" (UID: "ea698d13-b785-4a73-8563-04df09ce4424"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:54:28.398045 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:28.398020    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea698d13-b785-4a73-8563-04df09ce4424-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ea698d13-b785-4a73-8563-04df09ce4424" (UID: "ea698d13-b785-4a73-8563-04df09ce4424"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:54:28.398129 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:28.398046    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea698d13-b785-4a73-8563-04df09ce4424-kube-api-access-7jdjh" (OuterVolumeSpecName: "kube-api-access-7jdjh") pod "ea698d13-b785-4a73-8563-04df09ce4424" (UID: "ea698d13-b785-4a73-8563-04df09ce4424"). InnerVolumeSpecName "kube-api-access-7jdjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:54:28.497369 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:28.497334    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea698d13-b785-4a73-8563-04df09ce4424-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:28.497369 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:28.497366    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jdjh\" (UniqueName: \"kubernetes.io/projected/ea698d13-b785-4a73-8563-04df09ce4424-kube-api-access-7jdjh\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:28.497369 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:28.497376    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ea698d13-b785-4a73-8563-04df09ce4424-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:28.497602 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:28.497386    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea698d13-b785-4a73-8563-04df09ce4424-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:28.545029 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:28.544998    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb166992-c8bd-4419-945d-f18bd7726a2f" path="/var/lib/kubelet/pods/eb166992-c8bd-4419-945d-f18bd7726a2f/volumes"
Sep 18 06:54:28.737446 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:28.737355    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:29.110488 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:29.110396    2568 generic.go:356] "Generic (PLEG): container finished" podID="ea698d13-b785-4a73-8563-04df09ce4424" containerID="60811aded78ad3264eebbee762843413185aaefbda244276ba68e80d0e013bb8" exitCode=0
Sep 18 06:54:29.110488 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:29.110444    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"
Sep 18 06:54:29.110488 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:29.110478    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f" event={"ID":"ea698d13-b785-4a73-8563-04df09ce4424","Type":"ContainerDied","Data":"60811aded78ad3264eebbee762843413185aaefbda244276ba68e80d0e013bb8"}
Sep 18 06:54:29.111031 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:29.110541    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f" event={"ID":"ea698d13-b785-4a73-8563-04df09ce4424","Type":"ContainerDied","Data":"77eafbb1c78d4f7d9f5264ccd759e1ef01eb825f20f575fe5da8ba15db1285a6"}
Sep 18 06:54:29.111031 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:29.110558    2568 scope.go:117] "RemoveContainer" containerID="60811aded78ad3264eebbee762843413185aaefbda244276ba68e80d0e013bb8"
Sep 18 06:54:29.119983 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:29.119964    2568 scope.go:117] "RemoveContainer" containerID="60811aded78ad3264eebbee762843413185aaefbda244276ba68e80d0e013bb8"
Sep 18 06:54:29.120281 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:54:29.120260    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60811aded78ad3264eebbee762843413185aaefbda244276ba68e80d0e013bb8\": container with ID starting with 60811aded78ad3264eebbee762843413185aaefbda244276ba68e80d0e013bb8 not found: ID does not exist" containerID="60811aded78ad3264eebbee762843413185aaefbda244276ba68e80d0e013bb8"
Sep 18 06:54:29.120361 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:29.120288    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60811aded78ad3264eebbee762843413185aaefbda244276ba68e80d0e013bb8"} err="failed to get container status \"60811aded78ad3264eebbee762843413185aaefbda244276ba68e80d0e013bb8\": rpc error: code = NotFound desc = could not find container \"60811aded78ad3264eebbee762843413185aaefbda244276ba68e80d0e013bb8\": container with ID starting with 60811aded78ad3264eebbee762843413185aaefbda244276ba68e80d0e013bb8 not found: ID does not exist"
Sep 18 06:54:29.128022 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:29.127998    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"]
Sep 18 06:54:29.132710 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:29.132687    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-946609-9986b49d-c5r4f"]
Sep 18 06:54:29.539606 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:29.539559    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:29.808788 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:29.808700    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj" podUID="8a9ba24a-6256-4969-a7f2-d39d8d95cfc1" containerName="main" containerID="cri-o://bf96acc107bce0990422e8eaec71591832f5ab9e7b7fa3c735f62b97abee911f" gracePeriod=45
Sep 18 06:54:29.824328 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:29.824298    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6" podUID="1f9f4452-bbeb-4f35-880e-45ca235691b9" containerName="main" containerID="cri-o://39b222d48642db10fdfe134c9afba27c799f19933b6bc3f11308586aca67bef1" gracePeriod=45
Sep 18 06:54:29.843171 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:29.843135    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4" podUID="f64d9a93-2a33-412b-b470-77e030c9b071" containerName="main" containerID="cri-o://9c7688b8bf871f8fa93e78d13d75e5e96427e6cdd1316643a0a304c295be23fd" gracePeriod=45
Sep 18 06:54:29.863246 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:29.863206    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:29.940977 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:29.940934    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4" podUID="f64d9a93-2a33-412b-b470-77e030c9b071" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": dial tcp 10.133.0.54:8000: connect: connection refused"
Sep 18 06:54:29.963438 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:29.963411    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:30.123096 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.123061    2568 generic.go:356] "Generic (PLEG): container finished" podID="f64d9a93-2a33-412b-b470-77e030c9b071" containerID="9c7688b8bf871f8fa93e78d13d75e5e96427e6cdd1316643a0a304c295be23fd" exitCode=0
Sep 18 06:54:30.123572 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.123171    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4" event={"ID":"f64d9a93-2a33-412b-b470-77e030c9b071","Type":"ContainerDied","Data":"9c7688b8bf871f8fa93e78d13d75e5e96427e6cdd1316643a0a304c295be23fd"}
Sep 18 06:54:30.125441 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.125395    2568 generic.go:356] "Generic (PLEG): container finished" podID="1f9f4452-bbeb-4f35-880e-45ca235691b9" containerID="39b222d48642db10fdfe134c9afba27c799f19933b6bc3f11308586aca67bef1" exitCode=0
Sep 18 06:54:30.125588 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.125469    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6" event={"ID":"1f9f4452-bbeb-4f35-880e-45ca235691b9","Type":"ContainerDied","Data":"39b222d48642db10fdfe134c9afba27c799f19933b6bc3f11308586aca67bef1"}
Sep 18 06:54:30.128084 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.128051    2568 generic.go:356] "Generic (PLEG): container finished" podID="8a9ba24a-6256-4969-a7f2-d39d8d95cfc1" containerID="bf96acc107bce0990422e8eaec71591832f5ab9e7b7fa3c735f62b97abee911f" exitCode=0
Sep 18 06:54:30.128188 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.128110    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj" event={"ID":"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1","Type":"ContainerDied","Data":"bf96acc107bce0990422e8eaec71591832f5ab9e7b7fa3c735f62b97abee911f"}
Sep 18 06:54:30.140394 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.139196    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:30.174756 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.174730    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:54:30.178015 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.177990    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:54:30.181960 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.181944    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:54:30.215278 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.215249    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-model-cache\") pod \"f64d9a93-2a33-412b-b470-77e030c9b071\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") "
Sep 18 06:54:30.215471 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.215306    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgzhr\" (UniqueName: \"kubernetes.io/projected/f64d9a93-2a33-412b-b470-77e030c9b071-kube-api-access-vgzhr\") pod \"f64d9a93-2a33-412b-b470-77e030c9b071\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") "
Sep 18 06:54:30.215471 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.215332    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-dshm\") pod \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") "
Sep 18 06:54:30.215471 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.215361    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-home\") pod \"1f9f4452-bbeb-4f35-880e-45ca235691b9\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") "
Sep 18 06:54:30.215471 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.215393    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-model-cache\") pod \"1f9f4452-bbeb-4f35-880e-45ca235691b9\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") "
Sep 18 06:54:30.215471 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.215427    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-tmp-dir\") pod \"f64d9a93-2a33-412b-b470-77e030c9b071\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") "
Sep 18 06:54:30.215471 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.215442    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-dshm\") pod \"1f9f4452-bbeb-4f35-880e-45ca235691b9\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") "
Sep 18 06:54:30.215471 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.215462    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-tls-certs\") pod \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") "
Sep 18 06:54:30.215871 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.215491    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9mqb\" (UniqueName: \"kubernetes.io/projected/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-kube-api-access-w9mqb\") pod \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") "
Sep 18 06:54:30.215871 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.215513    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-home\") pod \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") "
Sep 18 06:54:30.215871 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.215528    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f64d9a93-2a33-412b-b470-77e030c9b071-tls-certs\") pod \"f64d9a93-2a33-412b-b470-77e030c9b071\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") "
Sep 18 06:54:30.215871 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.215547    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-dshm\") pod \"f64d9a93-2a33-412b-b470-77e030c9b071\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") "
Sep 18 06:54:30.215871 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.215552    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-model-cache" (OuterVolumeSpecName: "model-cache") pod "f64d9a93-2a33-412b-b470-77e030c9b071" (UID: "f64d9a93-2a33-412b-b470-77e030c9b071"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:30.215871 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.215598    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-tmp-dir\") pod \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") "
Sep 18 06:54:30.215871 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.215636    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-home\") pod \"f64d9a93-2a33-412b-b470-77e030c9b071\" (UID: \"f64d9a93-2a33-412b-b470-77e030c9b071\") "
Sep 18 06:54:30.215871 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.215710    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-tmp-dir\") pod \"1f9f4452-bbeb-4f35-880e-45ca235691b9\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") "
Sep 18 06:54:30.215871 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.215746    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-model-cache\") pod \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\" (UID: \"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1\") "
Sep 18 06:54:30.215871 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.215872    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f9f4452-bbeb-4f35-880e-45ca235691b9-tls-certs\") pod \"1f9f4452-bbeb-4f35-880e-45ca235691b9\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") "
Sep 18 06:54:30.216608 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.215910    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zn4x\" (UniqueName: \"kubernetes.io/projected/1f9f4452-bbeb-4f35-880e-45ca235691b9-kube-api-access-2zn4x\") pod \"1f9f4452-bbeb-4f35-880e-45ca235691b9\" (UID: \"1f9f4452-bbeb-4f35-880e-45ca235691b9\") "
Sep 18 06:54:30.216608 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.215987    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f64d9a93-2a33-412b-b470-77e030c9b071" (UID: "f64d9a93-2a33-412b-b470-77e030c9b071"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:30.216608 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.216211    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-home" (OuterVolumeSpecName: "home") pod "1f9f4452-bbeb-4f35-880e-45ca235691b9" (UID: "1f9f4452-bbeb-4f35-880e-45ca235691b9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:30.216608 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.216568    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:30.216608 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.216590    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:30.216608 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.216605    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:30.216918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.216816    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-model-cache" (OuterVolumeSpecName: "model-cache") pod "1f9f4452-bbeb-4f35-880e-45ca235691b9" (UID: "1f9f4452-bbeb-4f35-880e-45ca235691b9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:30.217807 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.216986    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1f9f4452-bbeb-4f35-880e-45ca235691b9" (UID: "1f9f4452-bbeb-4f35-880e-45ca235691b9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:30.217807 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.217162    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-home" (OuterVolumeSpecName: "home") pod "f64d9a93-2a33-412b-b470-77e030c9b071" (UID: "f64d9a93-2a33-412b-b470-77e030c9b071"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:30.217807 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.217264    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-model-cache" (OuterVolumeSpecName: "model-cache") pod "8a9ba24a-6256-4969-a7f2-d39d8d95cfc1" (UID: "8a9ba24a-6256-4969-a7f2-d39d8d95cfc1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:30.217807 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.217473    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-home" (OuterVolumeSpecName: "home") pod "8a9ba24a-6256-4969-a7f2-d39d8d95cfc1" (UID: "8a9ba24a-6256-4969-a7f2-d39d8d95cfc1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:30.217807 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.217547    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8a9ba24a-6256-4969-a7f2-d39d8d95cfc1" (UID: "8a9ba24a-6256-4969-a7f2-d39d8d95cfc1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:30.218373 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.218335    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-dshm" (OuterVolumeSpecName: "dshm") pod "f64d9a93-2a33-412b-b470-77e030c9b071" (UID: "f64d9a93-2a33-412b-b470-77e030c9b071"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:30.219779 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.219746    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f9f4452-bbeb-4f35-880e-45ca235691b9-kube-api-access-2zn4x" (OuterVolumeSpecName: "kube-api-access-2zn4x") pod "1f9f4452-bbeb-4f35-880e-45ca235691b9" (UID: "1f9f4452-bbeb-4f35-880e-45ca235691b9"). InnerVolumeSpecName "kube-api-access-2zn4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:54:30.220152 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.220060    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-dshm" (OuterVolumeSpecName: "dshm") pod "8a9ba24a-6256-4969-a7f2-d39d8d95cfc1" (UID: "8a9ba24a-6256-4969-a7f2-d39d8d95cfc1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:30.220152 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.220125    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-dshm" (OuterVolumeSpecName: "dshm") pod "1f9f4452-bbeb-4f35-880e-45ca235691b9" (UID: "1f9f4452-bbeb-4f35-880e-45ca235691b9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:30.220396 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.220368    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-kube-api-access-w9mqb" (OuterVolumeSpecName: "kube-api-access-w9mqb") pod "8a9ba24a-6256-4969-a7f2-d39d8d95cfc1" (UID: "8a9ba24a-6256-4969-a7f2-d39d8d95cfc1"). InnerVolumeSpecName "kube-api-access-w9mqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:54:30.220660 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.220613    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8a9ba24a-6256-4969-a7f2-d39d8d95cfc1" (UID: "8a9ba24a-6256-4969-a7f2-d39d8d95cfc1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:54:30.220660 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.220635    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f9f4452-bbeb-4f35-880e-45ca235691b9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1f9f4452-bbeb-4f35-880e-45ca235691b9" (UID: "1f9f4452-bbeb-4f35-880e-45ca235691b9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:54:30.221816 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.221789    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f64d9a93-2a33-412b-b470-77e030c9b071-kube-api-access-vgzhr" (OuterVolumeSpecName: "kube-api-access-vgzhr") pod "f64d9a93-2a33-412b-b470-77e030c9b071" (UID: "f64d9a93-2a33-412b-b470-77e030c9b071"). InnerVolumeSpecName "kube-api-access-vgzhr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:54:30.221939 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.221914    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f64d9a93-2a33-412b-b470-77e030c9b071-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f64d9a93-2a33-412b-b470-77e030c9b071" (UID: "f64d9a93-2a33-412b-b470-77e030c9b071"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:54:30.318106 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.318000    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:30.318106 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.318034    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f64d9a93-2a33-412b-b470-77e030c9b071-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:30.318106 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.318047    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:30.318106 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.318058    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:30.318106 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.318071    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f64d9a93-2a33-412b-b470-77e030c9b071-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:30.318106 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.318082    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:30.318106 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.318092    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:30.318106 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.318102    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f9f4452-bbeb-4f35-880e-45ca235691b9-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:30.318523 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.318115    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zn4x\" (UniqueName: \"kubernetes.io/projected/1f9f4452-bbeb-4f35-880e-45ca235691b9-kube-api-access-2zn4x\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:30.318523 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.318127    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgzhr\" (UniqueName: \"kubernetes.io/projected/f64d9a93-2a33-412b-b470-77e030c9b071-kube-api-access-vgzhr\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:30.318523 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.318139    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:30.318523 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.318151    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:30.318523 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.318160    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f9f4452-bbeb-4f35-880e-45ca235691b9-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:30.318523 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.318171    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:30.318523 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.318181    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9mqb\" (UniqueName: \"kubernetes.io/projected/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1-kube-api-access-w9mqb\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:30.422859 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.422823    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:30.544240 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:30.544197    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea698d13-b785-4a73-8563-04df09ce4424" path="/var/lib/kubelet/pods/ea698d13-b785-4a73-8563-04df09ce4424/volumes"
Sep 18 06:54:31.123485 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:31.123396    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:31.135942 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:31.135894    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"
Sep 18 06:54:31.135942 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:31.135908    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4" event={"ID":"f64d9a93-2a33-412b-b470-77e030c9b071","Type":"ContainerDied","Data":"af0b5bc468133df06e5389aa423321919ca4214e175e0acc50a8509016711957"}
Sep 18 06:54:31.136235 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:31.135968    2568 scope.go:117] "RemoveContainer" containerID="9c7688b8bf871f8fa93e78d13d75e5e96427e6cdd1316643a0a304c295be23fd"
Sep 18 06:54:31.138027 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:31.137921    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"
Sep 18 06:54:31.138027 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:31.137957    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6" event={"ID":"1f9f4452-bbeb-4f35-880e-45ca235691b9","Type":"ContainerDied","Data":"a0c2d10253ec00a00919464b7f8197905f3da29dbadb8d2f66c57b5f69baf071"}
Sep 18 06:54:31.139772 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:31.139752    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"
Sep 18 06:54:31.139772 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:31.139760    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj" event={"ID":"8a9ba24a-6256-4969-a7f2-d39d8d95cfc1","Type":"ContainerDied","Data":"b14f202e10c5b58052321d362c02f76094866995ce8d773ecbe777ad27c21fbd"}
Sep 18 06:54:31.146799 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:31.146780    2568 scope.go:117] "RemoveContainer" containerID="39b222d48642db10fdfe134c9afba27c799f19933b6bc3f11308586aca67bef1"
Sep 18 06:54:31.155328 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:31.155304    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"]
Sep 18 06:54:31.156071 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:31.156050    2568 scope.go:117] "RemoveContainer" containerID="bf96acc107bce0990422e8eaec71591832f5ab9e7b7fa3c735f62b97abee911f"
Sep 18 06:54:31.158085 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:31.158062    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-946609/distinct-2-w0-946609-kserve-fcf4c8cc8-gchpj"]
Sep 18 06:54:31.166581 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:31.166558    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"]
Sep 18 06:54:31.168649 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:31.168628    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-946609/embedding-w0-946609-kserve-748855cffd-fnjp6"]
Sep 18 06:54:31.180040 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:31.180017    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"]
Sep 18 06:54:31.185654 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:31.185632    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-946609/simulator-w0-946609-kserve-67569d6bcf-8n4m4"]
Sep 18 06:54:31.832377 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:31.832341    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:32.454613 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:32.454582    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:32.553166 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:32.553133    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f9f4452-bbeb-4f35-880e-45ca235691b9" path="/var/lib/kubelet/pods/1f9f4452-bbeb-4f35-880e-45ca235691b9/volumes"
Sep 18 06:54:32.553576 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:32.553558    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a9ba24a-6256-4969-a7f2-d39d8d95cfc1" path="/var/lib/kubelet/pods/8a9ba24a-6256-4969-a7f2-d39d8d95cfc1/volumes"
Sep 18 06:54:32.553933 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:32.553920    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f64d9a93-2a33-412b-b470-77e030c9b071" path="/var/lib/kubelet/pods/f64d9a93-2a33-412b-b470-77e030c9b071/volumes"
Sep 18 06:54:32.772248 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:32.772141    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:32.891129 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:32.891099    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-654463-z4rhv"]
Sep 18 06:54:32.891619 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:32.891603    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f9f4452-bbeb-4f35-880e-45ca235691b9" containerName="main"
Sep 18 06:54:32.891619 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:32.891621    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f9f4452-bbeb-4f35-880e-45ca235691b9" containerName="main"
Sep 18 06:54:32.891751 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:32.891634    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a9ba24a-6256-4969-a7f2-d39d8d95cfc1" containerName="main"
Sep 18 06:54:32.891751 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:32.891640    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a9ba24a-6256-4969-a7f2-d39d8d95cfc1" containerName="main"
Sep 18 06:54:32.891751 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:32.891652    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f64d9a93-2a33-412b-b470-77e030c9b071" containerName="main"
Sep 18 06:54:32.891751 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:32.891658    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f64d9a93-2a33-412b-b470-77e030c9b071" containerName="main"
Sep 18 06:54:32.891751 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:32.891666    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea698d13-b785-4a73-8563-04df09ce4424" containerName="maas-api"
Sep 18 06:54:32.891751 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:32.891671    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea698d13-b785-4a73-8563-04df09ce4424" containerName="maas-api"
Sep 18 06:54:32.891751 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:32.891736    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea698d13-b785-4a73-8563-04df09ce4424" containerName="maas-api"
Sep 18 06:54:32.891751 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:32.891745    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f64d9a93-2a33-412b-b470-77e030c9b071" containerName="main"
Sep 18 06:54:32.891751 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:32.891753    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a9ba24a-6256-4969-a7f2-d39d8d95cfc1" containerName="main"
Sep 18 06:54:32.892022 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:32.891759    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f9f4452-bbeb-4f35-880e-45ca235691b9" containerName="main"
Sep 18 06:54:32.896524 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:32.896506    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-654463-z4rhv"
Sep 18 06:54:32.899331 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:32.899308    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-88t7h\""
Sep 18 06:54:32.901522 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:32.901498    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-654463-z4rhv"]
Sep 18 06:54:32.944804 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:32.944769    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b25ada46-57da-4b48-be45-e518daced914-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-654463-z4rhv\" (UID: \"b25ada46-57da-4b48-be45-e518daced914\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-654463-z4rhv"
Sep 18 06:54:33.046188 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:33.046113    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b25ada46-57da-4b48-be45-e518daced914-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-654463-z4rhv\" (UID: \"b25ada46-57da-4b48-be45-e518daced914\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-654463-z4rhv"
Sep 18 06:54:33.046915 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:33.046889    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b25ada46-57da-4b48-be45-e518daced914-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-654463-z4rhv\" (UID: \"b25ada46-57da-4b48-be45-e518daced914\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-654463-z4rhv"
Sep 18 06:54:33.209716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:33.209680    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-654463-z4rhv"
Sep 18 06:54:33.412831 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:33.412794    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:33.484172 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:33.484143    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-654463-z4rhv"]
Sep 18 06:54:33.486302 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:54:33.486269    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb25ada46_57da_4b48_be45_e518daced914.slice/crio-3ca3f174297db93149cf3224dc8afd855fdcf12d106f83d25f7d00c3d9770562 WatchSource:0}: Error finding container 3ca3f174297db93149cf3224dc8afd855fdcf12d106f83d25f7d00c3d9770562: Status 404 returned error can't find the container with id 3ca3f174297db93149cf3224dc8afd855fdcf12d106f83d25f7d00c3d9770562
Sep 18 06:54:34.161180 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:34.161143    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-654463-z4rhv" event={"ID":"b25ada46-57da-4b48-be45-e518daced914","Type":"ContainerStarted","Data":"3192af24a8c4c714a1352f04c1a1ad5fb797f6c65b14979176e2cb4be134ffbf"}
Sep 18 06:54:34.161404 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:34.161205    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-654463-z4rhv" event={"ID":"b25ada46-57da-4b48-be45-e518daced914","Type":"ContainerStarted","Data":"3ca3f174297db93149cf3224dc8afd855fdcf12d106f83d25f7d00c3d9770562"}
Sep 18 06:54:34.190506 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:34.190457    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-654463-z4rhv" podStartSLOduration=2.190442249 podStartE2EDuration="2.190442249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:54:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:54:34.187318767 +0000 UTC m=+1939.868846512" watchObservedRunningTime="2026-09-18 06:54:34.190442249 +0000 UTC m=+1939.871970043"
Sep 18 06:54:34.256242 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:34.256131    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:34.344841 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:34.344798    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:34.467014 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:34.466931    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"]
Sep 18 06:54:34.467176 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:34.467145    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b" podUID="d88eebd0-4865-4103-ab85-92c6c00cbd74" containerName="istio-proxy" containerID="cri-o://014ee7475637d59e50fde608c06c418d3f4afef86c32fbf534648bc254472351" gracePeriod=30
Sep 18 06:54:34.494743 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:34.494710    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:35.079293 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:35.079258    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:35.169029 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:35.168993    2568 generic.go:356] "Generic (PLEG): container finished" podID="b25ada46-57da-4b48-be45-e518daced914" containerID="3192af24a8c4c714a1352f04c1a1ad5fb797f6c65b14979176e2cb4be134ffbf" exitCode=0
Sep 18 06:54:35.169204 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:35.169078    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-654463-z4rhv" event={"ID":"b25ada46-57da-4b48-be45-e518daced914","Type":"ContainerDied","Data":"3192af24a8c4c714a1352f04c1a1ad5fb797f6c65b14979176e2cb4be134ffbf"}
Sep 18 06:54:35.805970 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:35.805941    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:36.308261 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:36.308232    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-654463-z4rhv"
Sep 18 06:54:36.384368 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:36.384327    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b25ada46-57da-4b48-be45-e518daced914-maas-api-service-ca\") pod \"b25ada46-57da-4b48-be45-e518daced914\" (UID: \"b25ada46-57da-4b48-be45-e518daced914\") "
Sep 18 06:54:36.384675 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:36.384651    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b25ada46-57da-4b48-be45-e518daced914-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b25ada46-57da-4b48-be45-e518daced914" (UID: "b25ada46-57da-4b48-be45-e518daced914"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 06:54:36.435560 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:36.435474    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:36.486117 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:36.486077    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b25ada46-57da-4b48-be45-e518daced914-maas-api-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:37.063454 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:37.063412    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:37.181402 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:37.181373    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-654463-z4rhv"
Sep 18 06:54:37.181578 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:37.181412    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-654463-z4rhv" event={"ID":"b25ada46-57da-4b48-be45-e518daced914","Type":"ContainerDied","Data":"3ca3f174297db93149cf3224dc8afd855fdcf12d106f83d25f7d00c3d9770562"}
Sep 18 06:54:37.181578 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:37.181491    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ca3f174297db93149cf3224dc8afd855fdcf12d106f83d25f7d00c3d9770562"
Sep 18 06:54:37.763297 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:37.763262    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"]
Sep 18 06:54:37.779643 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:37.779614    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"]
Sep 18 06:54:37.796713 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:37.796676    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"]
Sep 18 06:54:37.810184 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:37.810156    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:37.866262 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:37.866212    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:38.787477 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:38.787437    2568 patch_prober.go:28] interesting pod/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 06:54:38.787930 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:38.787505    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b" podUID="d88eebd0-4865-4103-ab85-92c6c00cbd74" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 06:54:39.728181 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.728157    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:54:39.822024 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.821937    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-workload-socket\") pod \"d88eebd0-4865-4103-ab85-92c6c00cbd74\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") "
Sep 18 06:54:39.822024 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.821984    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-podinfo\") pod \"d88eebd0-4865-4103-ab85-92c6c00cbd74\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") "
Sep 18 06:54:39.822576 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.822030    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szrfs\" (UniqueName: \"kubernetes.io/projected/d88eebd0-4865-4103-ab85-92c6c00cbd74-kube-api-access-szrfs\") pod \"d88eebd0-4865-4103-ab85-92c6c00cbd74\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") "
Sep 18 06:54:39.822576 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.822049    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-workload-certs\") pod \"d88eebd0-4865-4103-ab85-92c6c00cbd74\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") "
Sep 18 06:54:39.822576 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.822086    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-credential-socket\") pod \"d88eebd0-4865-4103-ab85-92c6c00cbd74\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") "
Sep 18 06:54:39.822576 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.822119    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d88eebd0-4865-4103-ab85-92c6c00cbd74-istiod-ca-cert\") pod \"d88eebd0-4865-4103-ab85-92c6c00cbd74\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") "
Sep 18 06:54:39.822576 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.822175    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-token\") pod \"d88eebd0-4865-4103-ab85-92c6c00cbd74\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") "
Sep 18 06:54:39.822576 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.822245    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-data\") pod \"d88eebd0-4865-4103-ab85-92c6c00cbd74\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") "
Sep 18 06:54:39.822576 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.822278    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-envoy\") pod \"d88eebd0-4865-4103-ab85-92c6c00cbd74\" (UID: \"d88eebd0-4865-4103-ab85-92c6c00cbd74\") "
Sep 18 06:54:39.822576 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.822291    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d88eebd0-4865-4103-ab85-92c6c00cbd74" (UID: "d88eebd0-4865-4103-ab85-92c6c00cbd74"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:39.822576 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.822473    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d88eebd0-4865-4103-ab85-92c6c00cbd74" (UID: "d88eebd0-4865-4103-ab85-92c6c00cbd74"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:39.822576 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.822549    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-data" (OuterVolumeSpecName: "istio-data") pod "d88eebd0-4865-4103-ab85-92c6c00cbd74" (UID: "d88eebd0-4865-4103-ab85-92c6c00cbd74"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:39.823114 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.822658    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d88eebd0-4865-4103-ab85-92c6c00cbd74" (UID: "d88eebd0-4865-4103-ab85-92c6c00cbd74"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:39.823114 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.822678    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d88eebd0-4865-4103-ab85-92c6c00cbd74-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d88eebd0-4865-4103-ab85-92c6c00cbd74" (UID: "d88eebd0-4865-4103-ab85-92c6c00cbd74"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 06:54:39.823114 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.822792    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-credential-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:39.823114 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.822810    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d88eebd0-4865-4103-ab85-92c6c00cbd74-istiod-ca-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:39.823114 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.822824    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-data\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:39.823114 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.822836    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-workload-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:39.823114 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.822849    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-workload-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:39.824381 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.824347    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-token" (OuterVolumeSpecName: "istio-token") pod "d88eebd0-4865-4103-ab85-92c6c00cbd74" (UID: "d88eebd0-4865-4103-ab85-92c6c00cbd74"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:54:39.824492 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.824380    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d88eebd0-4865-4103-ab85-92c6c00cbd74" (UID: "d88eebd0-4865-4103-ab85-92c6c00cbd74"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 06:54:39.824492 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.824431    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d88eebd0-4865-4103-ab85-92c6c00cbd74" (UID: "d88eebd0-4865-4103-ab85-92c6c00cbd74"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:39.824581 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.824525    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d88eebd0-4865-4103-ab85-92c6c00cbd74-kube-api-access-szrfs" (OuterVolumeSpecName: "kube-api-access-szrfs") pod "d88eebd0-4865-4103-ab85-92c6c00cbd74" (UID: "d88eebd0-4865-4103-ab85-92c6c00cbd74"). InnerVolumeSpecName "kube-api-access-szrfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:54:39.924417 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.924376    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-podinfo\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:39.924417 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.924410    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szrfs\" (UniqueName: \"kubernetes.io/projected/d88eebd0-4865-4103-ab85-92c6c00cbd74-kube-api-access-szrfs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:39.924417 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.924424    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-token\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:39.924674 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:39.924435    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d88eebd0-4865-4103-ab85-92c6c00cbd74-istio-envoy\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:40.199873 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:40.199839    2568 generic.go:356] "Generic (PLEG): container finished" podID="d88eebd0-4865-4103-ab85-92c6c00cbd74" containerID="014ee7475637d59e50fde608c06c418d3f4afef86c32fbf534648bc254472351" exitCode=0
Sep 18 06:54:40.200055 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:40.199889    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"
Sep 18 06:54:40.200055 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:40.199899    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b" event={"ID":"d88eebd0-4865-4103-ab85-92c6c00cbd74","Type":"ContainerDied","Data":"014ee7475637d59e50fde608c06c418d3f4afef86c32fbf534648bc254472351"}
Sep 18 06:54:40.200055 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:40.199948    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b" event={"ID":"d88eebd0-4865-4103-ab85-92c6c00cbd74","Type":"ContainerDied","Data":"e47ae867012b597ef2349d9dac4da6c5f0702db047a84b3cf5a03fe647fbac81"}
Sep 18 06:54:40.200055 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:40.199972    2568 scope.go:117] "RemoveContainer" containerID="014ee7475637d59e50fde608c06c418d3f4afef86c32fbf534648bc254472351"
Sep 18 06:54:40.210438 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:40.210421    2568 scope.go:117] "RemoveContainer" containerID="214a428712c0451bd823faa0cb3a7739e7cd83185d5cc65d3d03bedbead4be01"
Sep 18 06:54:40.219114 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:40.219053    2568 scope.go:117] "RemoveContainer" containerID="014ee7475637d59e50fde608c06c418d3f4afef86c32fbf534648bc254472351"
Sep 18 06:54:40.219428 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:54:40.219401    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"014ee7475637d59e50fde608c06c418d3f4afef86c32fbf534648bc254472351\": container with ID starting with 014ee7475637d59e50fde608c06c418d3f4afef86c32fbf534648bc254472351 not found: ID does not exist" containerID="014ee7475637d59e50fde608c06c418d3f4afef86c32fbf534648bc254472351"
Sep 18 06:54:40.219509 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:40.219440    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"014ee7475637d59e50fde608c06c418d3f4afef86c32fbf534648bc254472351"} err="failed to get container status \"014ee7475637d59e50fde608c06c418d3f4afef86c32fbf534648bc254472351\": rpc error: code = NotFound desc = could not find container \"014ee7475637d59e50fde608c06c418d3f4afef86c32fbf534648bc254472351\": container with ID starting with 014ee7475637d59e50fde608c06c418d3f4afef86c32fbf534648bc254472351 not found: ID does not exist"
Sep 18 06:54:40.219509 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:40.219457    2568 scope.go:117] "RemoveContainer" containerID="214a428712c0451bd823faa0cb3a7739e7cd83185d5cc65d3d03bedbead4be01"
Sep 18 06:54:40.219740 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:54:40.219723    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"214a428712c0451bd823faa0cb3a7739e7cd83185d5cc65d3d03bedbead4be01\": container with ID starting with 214a428712c0451bd823faa0cb3a7739e7cd83185d5cc65d3d03bedbead4be01 not found: ID does not exist" containerID="214a428712c0451bd823faa0cb3a7739e7cd83185d5cc65d3d03bedbead4be01"
Sep 18 06:54:40.219789 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:40.219746    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"214a428712c0451bd823faa0cb3a7739e7cd83185d5cc65d3d03bedbead4be01"} err="failed to get container status \"214a428712c0451bd823faa0cb3a7739e7cd83185d5cc65d3d03bedbead4be01\": rpc error: code = NotFound desc = could not find container \"214a428712c0451bd823faa0cb3a7739e7cd83185d5cc65d3d03bedbead4be01\": container with ID starting with 214a428712c0451bd823faa0cb3a7739e7cd83185d5cc65d3d03bedbead4be01 not found: ID does not exist"
Sep 18 06:54:40.221724 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:40.221700    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"]
Sep 18 06:54:40.224615 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:40.224591    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b"]
Sep 18 06:54:40.543674 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:40.543581    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d88eebd0-4865-4103-ab85-92c6c00cbd74" path="/var/lib/kubelet/pods/d88eebd0-4865-4103-ab85-92c6c00cbd74/volumes"
Sep 18 06:54:40.553201 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:40.553167    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:41.045857 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.045824    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"]
Sep 18 06:54:41.046346 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.046068    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr" podUID="a819377f-5ecf-4063-9e9d-bbc234fe06b9" containerName="maas-api" containerID="cri-o://f772243021110256239a242e0686878b0d7734b33740501034e6e01ebf5af814" gracePeriod=30
Sep 18 06:54:41.072646 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.072621    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:41.205502 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.205420    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:41.213984 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.213948    2568 generic.go:356] "Generic (PLEG): container finished" podID="a819377f-5ecf-4063-9e9d-bbc234fe06b9" containerID="f772243021110256239a242e0686878b0d7734b33740501034e6e01ebf5af814" exitCode=0
Sep 18 06:54:41.214166 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.213988    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr" event={"ID":"a819377f-5ecf-4063-9e9d-bbc234fe06b9","Type":"ContainerDied","Data":"f772243021110256239a242e0686878b0d7734b33740501034e6e01ebf5af814"}
Sep 18 06:54:41.306820 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.306760    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"
Sep 18 06:54:41.309052 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.309024    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:41.406312 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.406279    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:41.441494 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.441457    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a819377f-5ecf-4063-9e9d-bbc234fe06b9-openshift-service-ca\") pod \"a819377f-5ecf-4063-9e9d-bbc234fe06b9\" (UID: \"a819377f-5ecf-4063-9e9d-bbc234fe06b9\") "
Sep 18 06:54:41.442474 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.441543    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a819377f-5ecf-4063-9e9d-bbc234fe06b9-metrics-certs\") pod \"a819377f-5ecf-4063-9e9d-bbc234fe06b9\" (UID: \"a819377f-5ecf-4063-9e9d-bbc234fe06b9\") "
Sep 18 06:54:41.442474 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.441581    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a819377f-5ecf-4063-9e9d-bbc234fe06b9-maas-api-tls\") pod \"a819377f-5ecf-4063-9e9d-bbc234fe06b9\" (UID: \"a819377f-5ecf-4063-9e9d-bbc234fe06b9\") "
Sep 18 06:54:41.442474 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.441650    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sshzq\" (UniqueName: \"kubernetes.io/projected/a819377f-5ecf-4063-9e9d-bbc234fe06b9-kube-api-access-sshzq\") pod \"a819377f-5ecf-4063-9e9d-bbc234fe06b9\" (UID: \"a819377f-5ecf-4063-9e9d-bbc234fe06b9\") "
Sep 18 06:54:41.442474 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.441886    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a819377f-5ecf-4063-9e9d-bbc234fe06b9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a819377f-5ecf-4063-9e9d-bbc234fe06b9" (UID: "a819377f-5ecf-4063-9e9d-bbc234fe06b9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 06:54:41.442474 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.442077    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a819377f-5ecf-4063-9e9d-bbc234fe06b9-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:41.444476 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.444436    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a819377f-5ecf-4063-9e9d-bbc234fe06b9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a819377f-5ecf-4063-9e9d-bbc234fe06b9" (UID: "a819377f-5ecf-4063-9e9d-bbc234fe06b9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:54:41.444625 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.444557    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a819377f-5ecf-4063-9e9d-bbc234fe06b9-kube-api-access-sshzq" (OuterVolumeSpecName: "kube-api-access-sshzq") pod "a819377f-5ecf-4063-9e9d-bbc234fe06b9" (UID: "a819377f-5ecf-4063-9e9d-bbc234fe06b9"). InnerVolumeSpecName "kube-api-access-sshzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:54:41.445158 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.445125    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a819377f-5ecf-4063-9e9d-bbc234fe06b9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a819377f-5ecf-4063-9e9d-bbc234fe06b9" (UID: "a819377f-5ecf-4063-9e9d-bbc234fe06b9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:54:41.542899 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.542867    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sshzq\" (UniqueName: \"kubernetes.io/projected/a819377f-5ecf-4063-9e9d-bbc234fe06b9-kube-api-access-sshzq\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:41.542899 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.542894    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a819377f-5ecf-4063-9e9d-bbc234fe06b9-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:41.542899 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.542903    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a819377f-5ecf-4063-9e9d-bbc234fe06b9-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:41.656688 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.656620    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:41.757152 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:41.757099    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:42.220387 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:42.220356    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"
Sep 18 06:54:42.220387 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:42.220372    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr" event={"ID":"a819377f-5ecf-4063-9e9d-bbc234fe06b9","Type":"ContainerDied","Data":"86e33b11830bc4ccd78e0703e17546cf5c6aff49e047b78a9a8113c9e701780c"}
Sep 18 06:54:42.220850 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:42.220439    2568 scope.go:117] "RemoveContainer" containerID="f772243021110256239a242e0686878b0d7734b33740501034e6e01ebf5af814"
Sep 18 06:54:42.246247 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:42.246208    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"]
Sep 18 06:54:42.248242 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:42.248199    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bac224-65c898987f-pgcjr"]
Sep 18 06:54:42.260801 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:54:42.260774    2568 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda819377f_5ecf_4063_9e9d_bbc234fe06b9.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda819377f_5ecf_4063_9e9d_bbc234fe06b9.slice/crio-86e33b11830bc4ccd78e0703e17546cf5c6aff49e047b78a9a8113c9e701780c\": RecentStats: unable to find data in memory cache]"
Sep 18 06:54:42.329036 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:42.329008    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:42.543690 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:42.543609    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a819377f-5ecf-4063-9e9d-bbc234fe06b9" path="/var/lib/kubelet/pods/a819377f-5ecf-4063-9e9d-bbc234fe06b9/volumes"
Sep 18 06:54:43.155576 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:43.155540    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:43.833858 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:43.833817    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:47.808464 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:47.808433    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:47.908959 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:47.908924    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:48.077837 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:48.077756    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:48.746453 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:48.746419    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:49.427101 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:49.427070    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:49.953745 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:49.953716    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:50.055363 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:50.055333    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:50.154918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:50.154866    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:50.257606 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:50.257525    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:50.287687 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:50.287655    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-654463-z4rhv"]
Sep 18 06:54:50.296778 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:50.296752    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-654463-z4rhv"]
Sep 18 06:54:50.357586 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:50.357556    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:50.543399 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:50.543326    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b25ada46-57da-4b48-be45-e518daced914" path="/var/lib/kubelet/pods/b25ada46-57da-4b48-be45-e518daced914/volumes"
Sep 18 06:54:50.644424 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:50.644392    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:51.077789 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:51.077726    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:51.200764 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:51.200709    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:51.302068 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:51.302038    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:51.920208 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:51.920178    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:52.567807 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:52.567774    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:52.782957 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:52.782911    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48" podUID="72047399-e64a-4137-b4dd-37b8bfcc32c1" containerName="main" containerID="cri-o://92c467d08c6528434cbb1d4e62f086db8f4f8887dc856bf2c4f184271ba525b5" gracePeriod=45
Sep 18 06:54:52.807622 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:52.807587    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw" podUID="287848f9-706a-4771-b889-99e2e1247bb8" containerName="main" containerID="cri-o://766ead08a0015fe7dd06ae09b7cabced4edaab384dc7b6e501bfe0c94c6a8e9d" gracePeriod=45
Sep 18 06:54:52.816107 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:52.816070    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95" podUID="871eb64e-4369-4bb5-915f-8c00f2adbbd1" containerName="main" containerID="cri-o://f0a9a5e63e0a674d4fd37f895d5e0ccc0d3602146ae8fc174227585c16d80d98" gracePeriod=45
Sep 18 06:54:53.043578 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.043192    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:53.111317 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.111248    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:54:53.114213 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.114191    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:54:53.118429 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.118409    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:54:53.247761 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.247707    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:53.265558 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.265530    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-dshm\") pod \"72047399-e64a-4137-b4dd-37b8bfcc32c1\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") "
Sep 18 06:54:53.265716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.265562    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-tmp-dir\") pod \"287848f9-706a-4771-b889-99e2e1247bb8\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") "
Sep 18 06:54:53.265716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.265591    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-tmp-dir\") pod \"72047399-e64a-4137-b4dd-37b8bfcc32c1\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") "
Sep 18 06:54:53.265716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.265627    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/287848f9-706a-4771-b889-99e2e1247bb8-tls-certs\") pod \"287848f9-706a-4771-b889-99e2e1247bb8\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") "
Sep 18 06:54:53.265716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.265642    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-home\") pod \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") "
Sep 18 06:54:53.265716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.265661    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xrr9m\" (UniqueName: \"kubernetes.io/projected/871eb64e-4369-4bb5-915f-8c00f2adbbd1-kube-api-access-xrr9m\") pod \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") "
Sep 18 06:54:53.265716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.265684    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-model-cache\") pod \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") "
Sep 18 06:54:53.265716 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.265712    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/871eb64e-4369-4bb5-915f-8c00f2adbbd1-tls-certs\") pod \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") "
Sep 18 06:54:53.266191 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.265734    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-tmp-dir\") pod \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") "
Sep 18 06:54:53.266191 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.265805    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v4gs\" (UniqueName: \"kubernetes.io/projected/287848f9-706a-4771-b889-99e2e1247bb8-kube-api-access-6v4gs\") pod \"287848f9-706a-4771-b889-99e2e1247bb8\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") "
Sep 18 06:54:53.266191 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.265834    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72047399-e64a-4137-b4dd-37b8bfcc32c1-tls-certs\") pod \"72047399-e64a-4137-b4dd-37b8bfcc32c1\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") "
Sep 18 06:54:53.266191 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.265831    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "287848f9-706a-4771-b889-99e2e1247bb8" (UID: "287848f9-706a-4771-b889-99e2e1247bb8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:53.266191 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.265861    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-model-cache\") pod \"287848f9-706a-4771-b889-99e2e1247bb8\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") "
Sep 18 06:54:53.266191 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.265941    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7f86f\" (UniqueName: \"kubernetes.io/projected/72047399-e64a-4137-b4dd-37b8bfcc32c1-kube-api-access-7f86f\") pod \"72047399-e64a-4137-b4dd-37b8bfcc32c1\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") "
Sep 18 06:54:53.266191 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.265952    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-model-cache" (OuterVolumeSpecName: "model-cache") pod "871eb64e-4369-4bb5-915f-8c00f2adbbd1" (UID: "871eb64e-4369-4bb5-915f-8c00f2adbbd1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:53.266191 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.265973    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-home\") pod \"72047399-e64a-4137-b4dd-37b8bfcc32c1\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") "
Sep 18 06:54:53.266191 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.266021    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-home\") pod \"287848f9-706a-4771-b889-99e2e1247bb8\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") "
Sep 18 06:54:53.266191 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.266040    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-model-cache" (OuterVolumeSpecName: "model-cache") pod "287848f9-706a-4771-b889-99e2e1247bb8" (UID: "287848f9-706a-4771-b889-99e2e1247bb8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:53.266191 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.266056    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-home" (OuterVolumeSpecName: "home") pod "871eb64e-4369-4bb5-915f-8c00f2adbbd1" (UID: "871eb64e-4369-4bb5-915f-8c00f2adbbd1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:53.266191 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.266149    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "72047399-e64a-4137-b4dd-37b8bfcc32c1" (UID: "72047399-e64a-4137-b4dd-37b8bfcc32c1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:53.266191 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.266164    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-home" (OuterVolumeSpecName: "home") pod "72047399-e64a-4137-b4dd-37b8bfcc32c1" (UID: "72047399-e64a-4137-b4dd-37b8bfcc32c1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:53.266902 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.266277    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-home" (OuterVolumeSpecName: "home") pod "287848f9-706a-4771-b889-99e2e1247bb8" (UID: "287848f9-706a-4771-b889-99e2e1247bb8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:53.266902 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.266291    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-dshm\") pod \"287848f9-706a-4771-b889-99e2e1247bb8\" (UID: \"287848f9-706a-4771-b889-99e2e1247bb8\") "
Sep 18 06:54:53.266902 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.266347    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-model-cache\") pod \"72047399-e64a-4137-b4dd-37b8bfcc32c1\" (UID: \"72047399-e64a-4137-b4dd-37b8bfcc32c1\") "
Sep 18 06:54:53.266902 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.266372    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-dshm\") pod \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\" (UID: \"871eb64e-4369-4bb5-915f-8c00f2adbbd1\") "
Sep 18 06:54:53.266902 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.266751    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "871eb64e-4369-4bb5-915f-8c00f2adbbd1" (UID: "871eb64e-4369-4bb5-915f-8c00f2adbbd1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:53.266902 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.266791    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:53.266902 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.266813    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:53.266902 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.266826    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:53.266902 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.266837    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:53.266902 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.266850    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:53.266902 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.266861    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:53.266902 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.266872    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:53.267542 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.267403    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-model-cache" (OuterVolumeSpecName: "model-cache") pod "72047399-e64a-4137-b4dd-37b8bfcc32c1" (UID: "72047399-e64a-4137-b4dd-37b8bfcc32c1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:53.268190 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.268160    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-dshm" (OuterVolumeSpecName: "dshm") pod "72047399-e64a-4137-b4dd-37b8bfcc32c1" (UID: "72047399-e64a-4137-b4dd-37b8bfcc32c1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:53.268392 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.268359    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/871eb64e-4369-4bb5-915f-8c00f2adbbd1-kube-api-access-xrr9m" (OuterVolumeSpecName: "kube-api-access-xrr9m") pod "871eb64e-4369-4bb5-915f-8c00f2adbbd1" (UID: "871eb64e-4369-4bb5-915f-8c00f2adbbd1"). InnerVolumeSpecName "kube-api-access-xrr9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:54:53.268392 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.268358    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/871eb64e-4369-4bb5-915f-8c00f2adbbd1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "871eb64e-4369-4bb5-915f-8c00f2adbbd1" (UID: "871eb64e-4369-4bb5-915f-8c00f2adbbd1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:54:53.268591 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.268440    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/287848f9-706a-4771-b889-99e2e1247bb8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "287848f9-706a-4771-b889-99e2e1247bb8" (UID: "287848f9-706a-4771-b889-99e2e1247bb8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:54:53.268871 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.268849    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/287848f9-706a-4771-b889-99e2e1247bb8-kube-api-access-6v4gs" (OuterVolumeSpecName: "kube-api-access-6v4gs") pod "287848f9-706a-4771-b889-99e2e1247bb8" (UID: "287848f9-706a-4771-b889-99e2e1247bb8"). InnerVolumeSpecName "kube-api-access-6v4gs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:54:53.269388 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.269357    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72047399-e64a-4137-b4dd-37b8bfcc32c1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "72047399-e64a-4137-b4dd-37b8bfcc32c1" (UID: "72047399-e64a-4137-b4dd-37b8bfcc32c1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:54:53.269599 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.269582    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72047399-e64a-4137-b4dd-37b8bfcc32c1-kube-api-access-7f86f" (OuterVolumeSpecName: "kube-api-access-7f86f") pod "72047399-e64a-4137-b4dd-37b8bfcc32c1" (UID: "72047399-e64a-4137-b4dd-37b8bfcc32c1"). InnerVolumeSpecName "kube-api-access-7f86f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:54:53.269653 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.269613    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-dshm" (OuterVolumeSpecName: "dshm") pod "871eb64e-4369-4bb5-915f-8c00f2adbbd1" (UID: "871eb64e-4369-4bb5-915f-8c00f2adbbd1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:53.270026 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.270003    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-dshm" (OuterVolumeSpecName: "dshm") pod "287848f9-706a-4771-b889-99e2e1247bb8" (UID: "287848f9-706a-4771-b889-99e2e1247bb8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:54:53.286359 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.286333    2568 generic.go:356] "Generic (PLEG): container finished" podID="72047399-e64a-4137-b4dd-37b8bfcc32c1" containerID="92c467d08c6528434cbb1d4e62f086db8f4f8887dc856bf2c4f184271ba525b5" exitCode=0
Sep 18 06:54:53.286468 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.286378    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"
Sep 18 06:54:53.286468 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.286410    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48" event={"ID":"72047399-e64a-4137-b4dd-37b8bfcc32c1","Type":"ContainerDied","Data":"92c467d08c6528434cbb1d4e62f086db8f4f8887dc856bf2c4f184271ba525b5"}
Sep 18 06:54:53.286468 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.286459    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48" event={"ID":"72047399-e64a-4137-b4dd-37b8bfcc32c1","Type":"ContainerDied","Data":"0adde68dfc2820f0facd75195a1e2c535c51efad8a9803c32eb3a875374c54ca"}
Sep 18 06:54:53.286625 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.286481    2568 scope.go:117] "RemoveContainer" containerID="92c467d08c6528434cbb1d4e62f086db8f4f8887dc856bf2c4f184271ba525b5"
Sep 18 06:54:53.287893 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.287868    2568 generic.go:356] "Generic (PLEG): container finished" podID="871eb64e-4369-4bb5-915f-8c00f2adbbd1" containerID="f0a9a5e63e0a674d4fd37f895d5e0ccc0d3602146ae8fc174227585c16d80d98" exitCode=0
Sep 18 06:54:53.287893 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.287883    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95" event={"ID":"871eb64e-4369-4bb5-915f-8c00f2adbbd1","Type":"ContainerDied","Data":"f0a9a5e63e0a674d4fd37f895d5e0ccc0d3602146ae8fc174227585c16d80d98"}
Sep 18 06:54:53.288054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.287920    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95" event={"ID":"871eb64e-4369-4bb5-915f-8c00f2adbbd1","Type":"ContainerDied","Data":"0b6be33758e38d00355e681e2304bcff993cff946bf88124a91b990768979e48"}
Sep 18 06:54:53.288054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.287879    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"
Sep 18 06:54:53.289508 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.289488    2568 generic.go:356] "Generic (PLEG): container finished" podID="287848f9-706a-4771-b889-99e2e1247bb8" containerID="766ead08a0015fe7dd06ae09b7cabced4edaab384dc7b6e501bfe0c94c6a8e9d" exitCode=0
Sep 18 06:54:53.289607 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.289537    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw" event={"ID":"287848f9-706a-4771-b889-99e2e1247bb8","Type":"ContainerDied","Data":"766ead08a0015fe7dd06ae09b7cabced4edaab384dc7b6e501bfe0c94c6a8e9d"}
Sep 18 06:54:53.289607 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.289565    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw" event={"ID":"287848f9-706a-4771-b889-99e2e1247bb8","Type":"ContainerDied","Data":"7ffeefbeffa9cf395324362c39cf1cdacc56690520d3c9f115016c19a65008a9"}
Sep 18 06:54:53.289607 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.289599    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"
Sep 18 06:54:53.296382 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.296365    2568 scope.go:117] "RemoveContainer" containerID="92c467d08c6528434cbb1d4e62f086db8f4f8887dc856bf2c4f184271ba525b5"
Sep 18 06:54:53.296689 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:54:53.296668    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92c467d08c6528434cbb1d4e62f086db8f4f8887dc856bf2c4f184271ba525b5\": container with ID starting with 92c467d08c6528434cbb1d4e62f086db8f4f8887dc856bf2c4f184271ba525b5 not found: ID does not exist" containerID="92c467d08c6528434cbb1d4e62f086db8f4f8887dc856bf2c4f184271ba525b5"
Sep 18 06:54:53.296776 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.296695    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92c467d08c6528434cbb1d4e62f086db8f4f8887dc856bf2c4f184271ba525b5"} err="failed to get container status \"92c467d08c6528434cbb1d4e62f086db8f4f8887dc856bf2c4f184271ba525b5\": rpc error: code = NotFound desc = could not find container \"92c467d08c6528434cbb1d4e62f086db8f4f8887dc856bf2c4f184271ba525b5\": container with ID starting with 92c467d08c6528434cbb1d4e62f086db8f4f8887dc856bf2c4f184271ba525b5 not found: ID does not exist"
Sep 18 06:54:53.296776 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.296711    2568 scope.go:117] "RemoveContainer" containerID="f0a9a5e63e0a674d4fd37f895d5e0ccc0d3602146ae8fc174227585c16d80d98"
Sep 18 06:54:53.305895 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.305881    2568 scope.go:117] "RemoveContainer" containerID="f0a9a5e63e0a674d4fd37f895d5e0ccc0d3602146ae8fc174227585c16d80d98"
Sep 18 06:54:53.306136 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:54:53.306117    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0a9a5e63e0a674d4fd37f895d5e0ccc0d3602146ae8fc174227585c16d80d98\": container with ID starting with f0a9a5e63e0a674d4fd37f895d5e0ccc0d3602146ae8fc174227585c16d80d98 not found: ID does not exist" containerID="f0a9a5e63e0a674d4fd37f895d5e0ccc0d3602146ae8fc174227585c16d80d98"
Sep 18 06:54:53.306195 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.306141    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0a9a5e63e0a674d4fd37f895d5e0ccc0d3602146ae8fc174227585c16d80d98"} err="failed to get container status \"f0a9a5e63e0a674d4fd37f895d5e0ccc0d3602146ae8fc174227585c16d80d98\": rpc error: code = NotFound desc = could not find container \"f0a9a5e63e0a674d4fd37f895d5e0ccc0d3602146ae8fc174227585c16d80d98\": container with ID starting with f0a9a5e63e0a674d4fd37f895d5e0ccc0d3602146ae8fc174227585c16d80d98 not found: ID does not exist"
Sep 18 06:54:53.306195 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.306156    2568 scope.go:117] "RemoveContainer" containerID="766ead08a0015fe7dd06ae09b7cabced4edaab384dc7b6e501bfe0c94c6a8e9d"
Sep 18 06:54:53.313000 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.312973    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"]
Sep 18 06:54:53.314552 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.314481    2568 scope.go:117] "RemoveContainer" containerID="766ead08a0015fe7dd06ae09b7cabced4edaab384dc7b6e501bfe0c94c6a8e9d"
Sep 18 06:54:53.314875 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:54:53.314849    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"766ead08a0015fe7dd06ae09b7cabced4edaab384dc7b6e501bfe0c94c6a8e9d\": container with ID starting with 766ead08a0015fe7dd06ae09b7cabced4edaab384dc7b6e501bfe0c94c6a8e9d not found: ID does not exist" containerID="766ead08a0015fe7dd06ae09b7cabced4edaab384dc7b6e501bfe0c94c6a8e9d"
Sep 18 06:54:53.314983 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.314885    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"766ead08a0015fe7dd06ae09b7cabced4edaab384dc7b6e501bfe0c94c6a8e9d"} err="failed to get container status \"766ead08a0015fe7dd06ae09b7cabced4edaab384dc7b6e501bfe0c94c6a8e9d\": rpc error: code = NotFound desc = could not find container \"766ead08a0015fe7dd06ae09b7cabced4edaab384dc7b6e501bfe0c94c6a8e9d\": container with ID starting with 766ead08a0015fe7dd06ae09b7cabced4edaab384dc7b6e501bfe0c94c6a8e9d not found: ID does not exist"
Sep 18 06:54:53.318255 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.318196    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-654463/distinct-2-w4-654463-kserve-55986bd45d-2sq48"]
Sep 18 06:54:53.330466 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.330441    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"]
Sep 18 06:54:53.333577 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.333558    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-654463/simulator-w4-654463-kserve-87b6f7bf9-flt95"]
Sep 18 06:54:53.343624 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.343602    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"]
Sep 18 06:54:53.347277 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.347257    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-654463/premium-w4-654463-kserve-6cbc644f8b-vj6xw"]
Sep 18 06:54:53.367631 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.367575    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/287848f9-706a-4771-b889-99e2e1247bb8-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:53.367631 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.367599    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrr9m\" (UniqueName: \"kubernetes.io/projected/871eb64e-4369-4bb5-915f-8c00f2adbbd1-kube-api-access-xrr9m\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:53.367631 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.367607    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/871eb64e-4369-4bb5-915f-8c00f2adbbd1-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:53.367631 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.367616    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:53.367631 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.367624    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v4gs\" (UniqueName: \"kubernetes.io/projected/287848f9-706a-4771-b889-99e2e1247bb8-kube-api-access-6v4gs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:53.367631 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.367632    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72047399-e64a-4137-b4dd-37b8bfcc32c1-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:53.367863 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.367641    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7f86f\" (UniqueName: \"kubernetes.io/projected/72047399-e64a-4137-b4dd-37b8bfcc32c1-kube-api-access-7f86f\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:53.367863 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.367649    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/287848f9-706a-4771-b889-99e2e1247bb8-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:53.367863 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.367657    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:53.367863 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.367664    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/871eb64e-4369-4bb5-915f-8c00f2adbbd1-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:53.367863 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:53.367670    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/72047399-e64a-4137-b4dd-37b8bfcc32c1-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:54:54.543728 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:54.543689    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="287848f9-706a-4771-b889-99e2e1247bb8" path="/var/lib/kubelet/pods/287848f9-706a-4771-b889-99e2e1247bb8/volumes"
Sep 18 06:54:54.544212 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:54.544083    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72047399-e64a-4137-b4dd-37b8bfcc32c1" path="/var/lib/kubelet/pods/72047399-e64a-4137-b4dd-37b8bfcc32c1/volumes"
Sep 18 06:54:54.544453 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:54.544438    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="871eb64e-4369-4bb5-915f-8c00f2adbbd1" path="/var/lib/kubelet/pods/871eb64e-4369-4bb5-915f-8c00f2adbbd1/volumes"
Sep 18 06:54:56.357994 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:56.357962    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:56.416107 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:56.416076    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:56.617509 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:56.617431    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:58.447421 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:58.447340    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:58.631705 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:58.631660    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:59.233042 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:59.233004    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:59.294160 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:59.294125    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:59.353589 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:59.353559    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:59.414041 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:59.414011    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:59.470496 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:59.470464    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:59.748477 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:59.748432    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:54:59.848802 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:54:59.848771    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:55:00.554077 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:00.554044    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:55:01.189699 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:01.189655    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:55:01.890625 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:01.890591    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:55:05.924774 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:05.924722    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:55:06.024651 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:06.024617    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:55:06.145505 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:06.145474    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"]
Sep 18 06:55:06.145736 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:06.145712    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn" podUID="52f56e47-435a-44dc-be93-b6cd22b2ce2e" containerName="istio-proxy" containerID="cri-o://6c959d28d5a56e6e156593ebe172bcedaf5f519c023e191a48cf20651b176e15" gracePeriod=30
Sep 18 06:55:06.170597 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:06.170574    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:55:09.452698 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:09.452660    2568 patch_prober.go:28] interesting pod/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 06:55:09.453063 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:09.452723    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn" podUID="52f56e47-435a-44dc-be93-b6cd22b2ce2e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 06:55:11.396487 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.396454    2568 generic.go:356] "Generic (PLEG): container finished" podID="52f56e47-435a-44dc-be93-b6cd22b2ce2e" containerID="6c959d28d5a56e6e156593ebe172bcedaf5f519c023e191a48cf20651b176e15" exitCode=0
Sep 18 06:55:11.396858 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.396529    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn" event={"ID":"52f56e47-435a-44dc-be93-b6cd22b2ce2e","Type":"ContainerDied","Data":"6c959d28d5a56e6e156593ebe172bcedaf5f519c023e191a48cf20651b176e15"}
Sep 18 06:55:11.396858 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.396571    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn" event={"ID":"52f56e47-435a-44dc-be93-b6cd22b2ce2e","Type":"ContainerDied","Data":"29c2c2899b32a319a368c17c4486be6dbd6455f9a87ddd2f5dd7918ea9674828"}
Sep 18 06:55:11.396858 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.396588    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="29c2c2899b32a319a368c17c4486be6dbd6455f9a87ddd2f5dd7918ea9674828"
Sep 18 06:55:11.396858 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.396601    2568 scope.go:117] "RemoveContainer" containerID="fb43b75821a5e34c7308716646637cf81fae806601a9489e52cf2c65f92088aa"
Sep 18 06:55:11.401662 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.401637    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:55:11.443015 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.442940    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-podinfo\") pod \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") "
Sep 18 06:55:11.443015 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.442992    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-envoy\") pod \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") "
Sep 18 06:55:11.443248 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.443024    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-workload-certs\") pod \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") "
Sep 18 06:55:11.443248 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.443043    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-credential-socket\") pod \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") "
Sep 18 06:55:11.443248 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.443084    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istiod-ca-cert\") pod \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") "
Sep 18 06:55:11.443248 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.443148    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-data\") pod \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") "
Sep 18 06:55:11.443248 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.443171    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-workload-socket\") pod \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") "
Sep 18 06:55:11.443248 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.443198    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zk9f2\" (UniqueName: \"kubernetes.io/projected/52f56e47-435a-44dc-be93-b6cd22b2ce2e-kube-api-access-zk9f2\") pod \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") "
Sep 18 06:55:11.443597 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.443269    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-token\") pod \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\" (UID: \"52f56e47-435a-44dc-be93-b6cd22b2ce2e\") "
Sep 18 06:55:11.443597 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.443366    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "52f56e47-435a-44dc-be93-b6cd22b2ce2e" (UID: "52f56e47-435a-44dc-be93-b6cd22b2ce2e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:55:11.443597 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.443381    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "52f56e47-435a-44dc-be93-b6cd22b2ce2e" (UID: "52f56e47-435a-44dc-be93-b6cd22b2ce2e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:55:11.443597 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.443444    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "52f56e47-435a-44dc-be93-b6cd22b2ce2e" (UID: "52f56e47-435a-44dc-be93-b6cd22b2ce2e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:55:11.443597 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.443541    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "52f56e47-435a-44dc-be93-b6cd22b2ce2e" (UID: "52f56e47-435a-44dc-be93-b6cd22b2ce2e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 06:55:11.443597 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.443576    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-data" (OuterVolumeSpecName: "istio-data") pod "52f56e47-435a-44dc-be93-b6cd22b2ce2e" (UID: "52f56e47-435a-44dc-be93-b6cd22b2ce2e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:55:11.443918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.443717    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-data\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:55:11.443918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.443740    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-workload-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:55:11.443918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.443755    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-workload-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:55:11.443918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.443770    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-credential-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:55:11.443918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.443785    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istiod-ca-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:55:11.445453 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.445423    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-token" (OuterVolumeSpecName: "istio-token") pod "52f56e47-435a-44dc-be93-b6cd22b2ce2e" (UID: "52f56e47-435a-44dc-be93-b6cd22b2ce2e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:55:11.445549 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.445532    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52f56e47-435a-44dc-be93-b6cd22b2ce2e-kube-api-access-zk9f2" (OuterVolumeSpecName: "kube-api-access-zk9f2") pod "52f56e47-435a-44dc-be93-b6cd22b2ce2e" (UID: "52f56e47-435a-44dc-be93-b6cd22b2ce2e"). InnerVolumeSpecName "kube-api-access-zk9f2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:55:11.445693 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.445671    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "52f56e47-435a-44dc-be93-b6cd22b2ce2e" (UID: "52f56e47-435a-44dc-be93-b6cd22b2ce2e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:55:11.445693 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.445677    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "52f56e47-435a-44dc-be93-b6cd22b2ce2e" (UID: "52f56e47-435a-44dc-be93-b6cd22b2ce2e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 06:55:11.544527 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.544491    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-envoy\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:55:11.544527 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.544526    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zk9f2\" (UniqueName: \"kubernetes.io/projected/52f56e47-435a-44dc-be93-b6cd22b2ce2e-kube-api-access-zk9f2\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:55:11.544722 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.544538    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-token\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:55:11.544722 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:11.544547    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52f56e47-435a-44dc-be93-b6cd22b2ce2e-istio-podinfo\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:55:12.404169 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:12.404145    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"
Sep 18 06:55:12.436095 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:12.436064    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"]
Sep 18 06:55:12.441376 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:12.441347    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn"]
Sep 18 06:55:12.543894 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:12.543860    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52f56e47-435a-44dc-be93-b6cd22b2ce2e" path="/var/lib/kubelet/pods/52f56e47-435a-44dc-be93-b6cd22b2ce2e/volumes"
Sep 18 06:55:14.050619 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:14.050581    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:55:14.150695 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:14.150653    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:55:15.362101 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:15.362067    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:55:16.057467 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:16.057434    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:55:16.158132 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:16.158099    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:55:16.742837 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:16.742806    2568 scope.go:117] "RemoveContainer" containerID="0ca6566058575d9d56fb741cc503f0cfe605aea078e1cbd385cff42df0dce63a"
Sep 18 06:55:16.751402 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:16.751380    2568 scope.go:117] "RemoveContainer" containerID="6c959d28d5a56e6e156593ebe172bcedaf5f519c023e191a48cf20651b176e15"
Sep 18 06:55:16.759885 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:16.759850    2568 scope.go:117] "RemoveContainer" containerID="c6dcd496bac232860829fbab928a0d443b3f099abf9212749ea02fe483126564"
Sep 18 06:55:18.088018 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:18.087986    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:55:21.055445 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:21.055406    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:55:28.334427 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.334388    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7756-151b6a"]
Sep 18 06:55:28.334812 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.334798    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b25ada46-57da-4b48-be45-e518daced914" containerName="revoke-keys"
Sep 18 06:55:28.334812 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.334810    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b25ada46-57da-4b48-be45-e518daced914" containerName="revoke-keys"
Sep 18 06:55:28.334886 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.334823    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d88eebd0-4865-4103-ab85-92c6c00cbd74" containerName="istio-proxy"
Sep 18 06:55:28.334886 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.334828    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d88eebd0-4865-4103-ab85-92c6c00cbd74" containerName="istio-proxy"
Sep 18 06:55:28.334886 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.334841    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="287848f9-706a-4771-b889-99e2e1247bb8" containerName="main"
Sep 18 06:55:28.334886 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.334847    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="287848f9-706a-4771-b889-99e2e1247bb8" containerName="main"
Sep 18 06:55:28.334886 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.334854    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52f56e47-435a-44dc-be93-b6cd22b2ce2e" containerName="istio-proxy"
Sep 18 06:55:28.334886 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.334859    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="52f56e47-435a-44dc-be93-b6cd22b2ce2e" containerName="istio-proxy"
Sep 18 06:55:28.334886 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.334870    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72047399-e64a-4137-b4dd-37b8bfcc32c1" containerName="main"
Sep 18 06:55:28.334886 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.334877    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="72047399-e64a-4137-b4dd-37b8bfcc32c1" containerName="main"
Sep 18 06:55:28.334886 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.334888    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="871eb64e-4369-4bb5-915f-8c00f2adbbd1" containerName="main"
Sep 18 06:55:28.335161 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.334893    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="871eb64e-4369-4bb5-915f-8c00f2adbbd1" containerName="main"
Sep 18 06:55:28.335161 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.334901    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d88eebd0-4865-4103-ab85-92c6c00cbd74" containerName="istio-proxy"
Sep 18 06:55:28.335161 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.334906    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d88eebd0-4865-4103-ab85-92c6c00cbd74" containerName="istio-proxy"
Sep 18 06:55:28.335161 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.334911    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52f56e47-435a-44dc-be93-b6cd22b2ce2e" containerName="istio-proxy"
Sep 18 06:55:28.335161 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.334917    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="52f56e47-435a-44dc-be93-b6cd22b2ce2e" containerName="istio-proxy"
Sep 18 06:55:28.335161 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.334926    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a819377f-5ecf-4063-9e9d-bbc234fe06b9" containerName="maas-api"
Sep 18 06:55:28.335161 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.334931    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a819377f-5ecf-4063-9e9d-bbc234fe06b9" containerName="maas-api"
Sep 18 06:55:28.335161 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.334985    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="287848f9-706a-4771-b889-99e2e1247bb8" containerName="main"
Sep 18 06:55:28.335161 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.334992    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="52f56e47-435a-44dc-be93-b6cd22b2ce2e" containerName="istio-proxy"
Sep 18 06:55:28.335161 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.335001    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b25ada46-57da-4b48-be45-e518daced914" containerName="revoke-keys"
Sep 18 06:55:28.335161 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.335008    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="871eb64e-4369-4bb5-915f-8c00f2adbbd1" containerName="main"
Sep 18 06:55:28.335161 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.335015    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a819377f-5ecf-4063-9e9d-bbc234fe06b9" containerName="maas-api"
Sep 18 06:55:28.335161 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.335021    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d88eebd0-4865-4103-ab85-92c6c00cbd74" containerName="istio-proxy"
Sep 18 06:55:28.335161 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.335028    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="52f56e47-435a-44dc-be93-b6cd22b2ce2e" containerName="istio-proxy"
Sep 18 06:55:28.335161 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.335032    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="72047399-e64a-4137-b4dd-37b8bfcc32c1" containerName="main"
Sep 18 06:55:28.335161 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.335038    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d88eebd0-4865-4103-ab85-92c6c00cbd74" containerName="istio-proxy"
Sep 18 06:55:28.340542 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.340511    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7756-151b6a"
Sep 18 06:55:28.342927 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.342902    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-h78xn\""
Sep 18 06:55:28.343493 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.343467    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7756-151b6a"]
Sep 18 06:55:28.507520 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.507481    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whvs6\" (UniqueName: \"kubernetes.io/projected/4d948e04-d481-42ec-800f-f533c0d8a06c-kube-api-access-whvs6\") pod \"test-curl-7756-151b6a\" (UID: \"4d948e04-d481-42ec-800f-f533c0d8a06c\") " pod="opendatahub/test-curl-7756-151b6a"
Sep 18 06:55:28.608406 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.608308    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whvs6\" (UniqueName: \"kubernetes.io/projected/4d948e04-d481-42ec-800f-f533c0d8a06c-kube-api-access-whvs6\") pod \"test-curl-7756-151b6a\" (UID: \"4d948e04-d481-42ec-800f-f533c0d8a06c\") " pod="opendatahub/test-curl-7756-151b6a"
Sep 18 06:55:28.617168 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.617143    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whvs6\" (UniqueName: \"kubernetes.io/projected/4d948e04-d481-42ec-800f-f533c0d8a06c-kube-api-access-whvs6\") pod \"test-curl-7756-151b6a\" (UID: \"4d948e04-d481-42ec-800f-f533c0d8a06c\") " pod="opendatahub/test-curl-7756-151b6a"
Sep 18 06:55:28.652458 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.652430    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7756-151b6a"
Sep 18 06:55:28.918205 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:28.918183    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7756-151b6a"]
Sep 18 06:55:28.920103 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:55:28.920077    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d948e04_d481_42ec_800f_f533c0d8a06c.slice/crio-8519df909d355f91256f5c4088c21e79de2bdfee5029b6bb4908b5332b47e1cf WatchSource:0}: Error finding container 8519df909d355f91256f5c4088c21e79de2bdfee5029b6bb4908b5332b47e1cf: Status 404 returned error can't find the container with id 8519df909d355f91256f5c4088c21e79de2bdfee5029b6bb4908b5332b47e1cf
Sep 18 06:55:29.501100 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:29.501055    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7756-151b6a" event={"ID":"4d948e04-d481-42ec-800f-f533c0d8a06c","Type":"ContainerStarted","Data":"b0b8b2187175986a365c893404906b3e6152d2f382f97e97276601f311ea0d6f"}
Sep 18 06:55:29.501530 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:29.501113    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7756-151b6a" event={"ID":"4d948e04-d481-42ec-800f-f533c0d8a06c","Type":"ContainerStarted","Data":"8519df909d355f91256f5c4088c21e79de2bdfee5029b6bb4908b5332b47e1cf"}
Sep 18 06:55:29.515102 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:29.515056    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7756-151b6a" podStartSLOduration=1.515040768 podStartE2EDuration="1.515040768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:55:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:55:29.513507896 +0000 UTC m=+1995.195035639" watchObservedRunningTime="2026-09-18 06:55:29.515040768 +0000 UTC m=+1995.196568512"
Sep 18 06:55:30.080759 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:30.080721    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7756-151b6a"]
Sep 18 06:55:30.082846 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:30.082820    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7756-151b6a"]
Sep 18 06:55:30.506571 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:55:30.506541    2568 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7756-151b6a" container="test-curl-7756-151b6a"
Sep 18 06:55:31.186984 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:31.186936    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:55:31.415549 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:31.415516    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:55:31.511129 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:31.511027    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7756-151b6a" podUID="4d948e04-d481-42ec-800f-f533c0d8a06c" containerName="test-curl-7756-151b6a" containerID="cri-o://b0b8b2187175986a365c893404906b3e6152d2f382f97e97276601f311ea0d6f" gracePeriod=2
Sep 18 06:55:32.394749 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:32.394716    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"]
Sep 18 06:55:32.406607 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:32.406575    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:55:33.686210 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:33.686187    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7756-151b6a"
Sep 18 06:55:33.862991 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:33.862913    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whvs6\" (UniqueName: \"kubernetes.io/projected/4d948e04-d481-42ec-800f-f533c0d8a06c-kube-api-access-whvs6\") pod \"4d948e04-d481-42ec-800f-f533c0d8a06c\" (UID: \"4d948e04-d481-42ec-800f-f533c0d8a06c\") "
Sep 18 06:55:33.864730 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:33.864698    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d948e04-d481-42ec-800f-f533c0d8a06c-kube-api-access-whvs6" (OuterVolumeSpecName: "kube-api-access-whvs6") pod "4d948e04-d481-42ec-800f-f533c0d8a06c" (UID: "4d948e04-d481-42ec-800f-f533c0d8a06c"). InnerVolumeSpecName "kube-api-access-whvs6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:55:33.964762 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:33.964730    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whvs6\" (UniqueName: \"kubernetes.io/projected/4d948e04-d481-42ec-800f-f533c0d8a06c-kube-api-access-whvs6\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:55:34.527577 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:34.527536    2568 generic.go:356] "Generic (PLEG): container finished" podID="4d948e04-d481-42ec-800f-f533c0d8a06c" containerID="b0b8b2187175986a365c893404906b3e6152d2f382f97e97276601f311ea0d6f" exitCode=137
Sep 18 06:55:34.527577 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:34.527580    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7756-151b6a"
Sep 18 06:55:34.527848 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:34.527641    2568 scope.go:117] "RemoveContainer" containerID="b0b8b2187175986a365c893404906b3e6152d2f382f97e97276601f311ea0d6f"
Sep 18 06:55:34.537876 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:55:34.537846    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7756-151b6a\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="4d948e04-d481-42ec-800f-f533c0d8a06c" pod="opendatahub/test-curl-7756-151b6a"
Sep 18 06:55:34.537952 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:34.537874    2568 scope.go:117] "RemoveContainer" containerID="b0b8b2187175986a365c893404906b3e6152d2f382f97e97276601f311ea0d6f"
Sep 18 06:55:34.538201 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:55:34.538183    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0b8b2187175986a365c893404906b3e6152d2f382f97e97276601f311ea0d6f\": container with ID starting with b0b8b2187175986a365c893404906b3e6152d2f382f97e97276601f311ea0d6f not found: ID does not exist" containerID="b0b8b2187175986a365c893404906b3e6152d2f382f97e97276601f311ea0d6f"
Sep 18 06:55:34.538276 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:34.538212    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0b8b2187175986a365c893404906b3e6152d2f382f97e97276601f311ea0d6f"} err="failed to get container status \"b0b8b2187175986a365c893404906b3e6152d2f382f97e97276601f311ea0d6f\": rpc error: code = NotFound desc = could not find container \"b0b8b2187175986a365c893404906b3e6152d2f382f97e97276601f311ea0d6f\": container with ID starting with b0b8b2187175986a365c893404906b3e6152d2f382f97e97276601f311ea0d6f not found: ID does not exist"
Sep 18 06:55:34.543293 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:34.543263    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d948e04-d481-42ec-800f-f533c0d8a06c" path="/var/lib/kubelet/pods/4d948e04-d481-42ec-800f-f533c0d8a06c/volumes"
Sep 18 06:55:41.588409 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:41.588372    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:55:47.414109 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:47.414068    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds" podUID="24eb3bee-e209-4fe4-a1df-095c77af875d" containerName="main" containerID="cri-o://71cec49caf841b30ee89cf83bcd367213628f4cfcac456356401841a0c83d32e" gracePeriod=45
Sep 18 06:55:47.600578 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:47.600548    2568 generic.go:356] "Generic (PLEG): container finished" podID="24eb3bee-e209-4fe4-a1df-095c77af875d" containerID="71cec49caf841b30ee89cf83bcd367213628f4cfcac456356401841a0c83d32e" exitCode=0
Sep 18 06:55:47.600721 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:47.600622    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds" event={"ID":"24eb3bee-e209-4fe4-a1df-095c77af875d","Type":"ContainerDied","Data":"71cec49caf841b30ee89cf83bcd367213628f4cfcac456356401841a0c83d32e"}
Sep 18 06:55:47.687381 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:47.687355    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:55:47.797656 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:47.797626    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-dshm\") pod \"24eb3bee-e209-4fe4-a1df-095c77af875d\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") "
Sep 18 06:55:47.797797 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:47.797690    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-home\") pod \"24eb3bee-e209-4fe4-a1df-095c77af875d\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") "
Sep 18 06:55:47.797797 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:47.797736    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-tmp-dir\") pod \"24eb3bee-e209-4fe4-a1df-095c77af875d\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") "
Sep 18 06:55:47.797797 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:47.797769    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24eb3bee-e209-4fe4-a1df-095c77af875d-tls-certs\") pod \"24eb3bee-e209-4fe4-a1df-095c77af875d\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") "
Sep 18 06:55:47.797987 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:47.797810    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5gth\" (UniqueName: \"kubernetes.io/projected/24eb3bee-e209-4fe4-a1df-095c77af875d-kube-api-access-s5gth\") pod \"24eb3bee-e209-4fe4-a1df-095c77af875d\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") "
Sep 18 06:55:47.797987 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:47.797838    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-model-cache\") pod \"24eb3bee-e209-4fe4-a1df-095c77af875d\" (UID: \"24eb3bee-e209-4fe4-a1df-095c77af875d\") "
Sep 18 06:55:47.797987 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:47.797882    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-home" (OuterVolumeSpecName: "home") pod "24eb3bee-e209-4fe4-a1df-095c77af875d" (UID: "24eb3bee-e209-4fe4-a1df-095c77af875d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:55:47.798167 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:47.798009    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "24eb3bee-e209-4fe4-a1df-095c77af875d" (UID: "24eb3bee-e209-4fe4-a1df-095c77af875d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:55:47.798167 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:47.798103    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-model-cache" (OuterVolumeSpecName: "model-cache") pod "24eb3bee-e209-4fe4-a1df-095c77af875d" (UID: "24eb3bee-e209-4fe4-a1df-095c77af875d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:55:47.798167 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:47.798130    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:55:47.798167 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:47.798146    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:55:47.799896 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:47.799876    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-dshm" (OuterVolumeSpecName: "dshm") pod "24eb3bee-e209-4fe4-a1df-095c77af875d" (UID: "24eb3bee-e209-4fe4-a1df-095c77af875d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:55:47.799955 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:47.799935    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24eb3bee-e209-4fe4-a1df-095c77af875d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "24eb3bee-e209-4fe4-a1df-095c77af875d" (UID: "24eb3bee-e209-4fe4-a1df-095c77af875d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:55:47.799991 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:47.799958    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24eb3bee-e209-4fe4-a1df-095c77af875d-kube-api-access-s5gth" (OuterVolumeSpecName: "kube-api-access-s5gth") pod "24eb3bee-e209-4fe4-a1df-095c77af875d" (UID: "24eb3bee-e209-4fe4-a1df-095c77af875d"). InnerVolumeSpecName "kube-api-access-s5gth". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:55:47.899596 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:47.899559    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/24eb3bee-e209-4fe4-a1df-095c77af875d-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:55:47.899596 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:47.899591    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5gth\" (UniqueName: \"kubernetes.io/projected/24eb3bee-e209-4fe4-a1df-095c77af875d-kube-api-access-s5gth\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:55:47.899596 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:47.899604    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:55:47.899818 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:47.899616    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/24eb3bee-e209-4fe4-a1df-095c77af875d-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:55:48.607002 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:48.606975    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"
Sep 18 06:55:48.607421 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:48.607001    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds" event={"ID":"24eb3bee-e209-4fe4-a1df-095c77af875d","Type":"ContainerDied","Data":"4264faf2056ff498609715c0dbe9c2f612acfa5cfb2edd6dfa8600076665f552"}
Sep 18 06:55:48.607421 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:48.607043    2568 scope.go:117] "RemoveContainer" containerID="71cec49caf841b30ee89cf83bcd367213628f4cfcac456356401841a0c83d32e"
Sep 18 06:55:48.622401 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:48.622378    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"]
Sep 18 06:55:48.625516 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:48.625493    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8fdd5b/auth-test-model-8fdd5b-kserve-658b55b847-jnnds"]
Sep 18 06:55:50.543034 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:50.542997    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24eb3bee-e209-4fe4-a1df-095c77af875d" path="/var/lib/kubelet/pods/24eb3bee-e209-4fe4-a1df-095c77af875d/volumes"
Sep 18 06:55:59.260786 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:55:59.260743    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:56:02.590104 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.590070    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"]
Sep 18 06:56:02.590597 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.590480    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24eb3bee-e209-4fe4-a1df-095c77af875d" containerName="main"
Sep 18 06:56:02.590597 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.590493    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="24eb3bee-e209-4fe4-a1df-095c77af875d" containerName="main"
Sep 18 06:56:02.590597 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.590502    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d948e04-d481-42ec-800f-f533c0d8a06c" containerName="test-curl-7756-151b6a"
Sep 18 06:56:02.590597 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.590508    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d948e04-d481-42ec-800f-f533c0d8a06c" containerName="test-curl-7756-151b6a"
Sep 18 06:56:02.590597 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.590559    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="24eb3bee-e209-4fe4-a1df-095c77af875d" containerName="main"
Sep 18 06:56:02.590597 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.590569    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d948e04-d481-42ec-800f-f533c0d8a06c" containerName="test-curl-7756-151b6a"
Sep 18 06:56:02.594826 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.594804    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:02.597246 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.597194    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ec2077\"/\"kube-root-ca.crt\""
Sep 18 06:56:02.598545 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.598523    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ec2077\"/\"sub-test-model-ec2077-kserve-self-signed-certs\""
Sep 18 06:56:02.598710 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.598530    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ec2077\"/\"default-dockercfg-b954k\""
Sep 18 06:56:02.598784 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.598586    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ec2077\"/\"openshift-service-ca.crt\""
Sep 18 06:56:02.601495 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.601471    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"]
Sep 18 06:56:02.611574 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.611547    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:56:02.744282 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.744245    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-tmp-dir\") pod \"sub-test-model-ec2077-kserve-6757d78567-7qlng\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:02.744458 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.744305    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-home\") pod \"sub-test-model-ec2077-kserve-6757d78567-7qlng\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:02.744458 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.744365    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-dshm\") pod \"sub-test-model-ec2077-kserve-6757d78567-7qlng\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:02.744549 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.744462    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpxbw\" (UniqueName: \"kubernetes.io/projected/9ff07573-2b58-456e-85c7-79974d9a5c94-kube-api-access-qpxbw\") pod \"sub-test-model-ec2077-kserve-6757d78567-7qlng\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:02.744549 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.744482    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-model-cache\") pod \"sub-test-model-ec2077-kserve-6757d78567-7qlng\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:02.744620 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.744570    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ff07573-2b58-456e-85c7-79974d9a5c94-tls-certs\") pod \"sub-test-model-ec2077-kserve-6757d78567-7qlng\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:02.845258 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.845155    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpxbw\" (UniqueName: \"kubernetes.io/projected/9ff07573-2b58-456e-85c7-79974d9a5c94-kube-api-access-qpxbw\") pod \"sub-test-model-ec2077-kserve-6757d78567-7qlng\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:02.845258 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.845190    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-model-cache\") pod \"sub-test-model-ec2077-kserve-6757d78567-7qlng\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:02.845258 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.845256    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ff07573-2b58-456e-85c7-79974d9a5c94-tls-certs\") pod \"sub-test-model-ec2077-kserve-6757d78567-7qlng\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:02.845548 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.845298    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-tmp-dir\") pod \"sub-test-model-ec2077-kserve-6757d78567-7qlng\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:02.845548 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.845348    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-home\") pod \"sub-test-model-ec2077-kserve-6757d78567-7qlng\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:02.845548 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.845367    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-dshm\") pod \"sub-test-model-ec2077-kserve-6757d78567-7qlng\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:02.845741 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.845711    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-model-cache\") pod \"sub-test-model-ec2077-kserve-6757d78567-7qlng\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:02.845832 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.845777    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-tmp-dir\") pod \"sub-test-model-ec2077-kserve-6757d78567-7qlng\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:02.845888 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.845841    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-home\") pod \"sub-test-model-ec2077-kserve-6757d78567-7qlng\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:02.847608 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.847589    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-dshm\") pod \"sub-test-model-ec2077-kserve-6757d78567-7qlng\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:02.847911 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.847892    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ff07573-2b58-456e-85c7-79974d9a5c94-tls-certs\") pod \"sub-test-model-ec2077-kserve-6757d78567-7qlng\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:02.852683 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.852655    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpxbw\" (UniqueName: \"kubernetes.io/projected/9ff07573-2b58-456e-85c7-79974d9a5c94-kube-api-access-qpxbw\") pod \"sub-test-model-ec2077-kserve-6757d78567-7qlng\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:02.906488 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:02.906446    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:03.180805 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:03.180776    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"]
Sep 18 06:56:03.182370 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:56:03.182341    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ff07573_2b58_456e_85c7_79974d9a5c94.slice/crio-6ffd3c48836e5b63e794e76204e51598470f01dd21d0d21b2d4ccf475ae44418 WatchSource:0}: Error finding container 6ffd3c48836e5b63e794e76204e51598470f01dd21d0d21b2d4ccf475ae44418: Status 404 returned error can't find the container with id 6ffd3c48836e5b63e794e76204e51598470f01dd21d0d21b2d4ccf475ae44418
Sep 18 06:56:03.695743 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:03.695712    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng" event={"ID":"9ff07573-2b58-456e-85c7-79974d9a5c94","Type":"ContainerStarted","Data":"0e3e3fe3c61e4c8d4c0d2b8afc7ea328282c7fc5a003f39d6b57fa7986474195"}
Sep 18 06:56:03.696131 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:03.695750    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng" event={"ID":"9ff07573-2b58-456e-85c7-79974d9a5c94","Type":"ContainerStarted","Data":"6ffd3c48836e5b63e794e76204e51598470f01dd21d0d21b2d4ccf475ae44418"}
Sep 18 06:56:03.713230 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:03.713165    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng" podStartSLOduration=1.713149016 podStartE2EDuration="1.713149016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:56:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:56:03.711209267 +0000 UTC m=+2029.392737011" watchObservedRunningTime="2026-09-18 06:56:03.713149016 +0000 UTC m=+2029.394676760"
Sep 18 06:56:05.693152 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:05.693115    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:56:12.907281 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:12.907241    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:12.907727 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:12.907299    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:12.919945 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:12.919917    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:13.758743 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:13.758715    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:56:17.573308 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.573275    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"]
Sep 18 06:56:17.578511 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.578486    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:17.582174 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.582151    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-8fdd5b\"/\"default-dockercfg-nl8gc\""
Sep 18 06:56:17.582433 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.582154    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-8fdd5b\"/\"kube-root-ca.crt\""
Sep 18 06:56:17.582586 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.582197    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-8fdd5b\"/\"openshift-service-ca.crt\""
Sep 18 06:56:17.582798 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.582765    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-8fdd5b\"/\"sub-test-model-8fdd5b-kserve-self-signed-certs\""
Sep 18 06:56:17.585934 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.585907    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"]
Sep 18 06:56:17.596107 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.596084    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:56:17.677941 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.677915    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-dshm\") pod \"sub-test-model-8fdd5b-kserve-6b965b5749-thzfq\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:17.678108 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.677970    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-home\") pod \"sub-test-model-8fdd5b-kserve-6b965b5749-thzfq\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:17.678108 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.678054    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f527a21-a030-49c3-86f8-dd6407bc3c23-tls-certs\") pod \"sub-test-model-8fdd5b-kserve-6b965b5749-thzfq\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:17.678108 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.678082    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-tmp-dir\") pod \"sub-test-model-8fdd5b-kserve-6b965b5749-thzfq\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:17.678242 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.678120    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7mkt\" (UniqueName: \"kubernetes.io/projected/2f527a21-a030-49c3-86f8-dd6407bc3c23-kube-api-access-k7mkt\") pod \"sub-test-model-8fdd5b-kserve-6b965b5749-thzfq\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:17.678242 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.678177    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-model-cache\") pod \"sub-test-model-8fdd5b-kserve-6b965b5749-thzfq\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:17.778907 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.778877    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-home\") pod \"sub-test-model-8fdd5b-kserve-6b965b5749-thzfq\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:17.779093 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.778918    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f527a21-a030-49c3-86f8-dd6407bc3c23-tls-certs\") pod \"sub-test-model-8fdd5b-kserve-6b965b5749-thzfq\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:17.779093 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.778935    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-tmp-dir\") pod \"sub-test-model-8fdd5b-kserve-6b965b5749-thzfq\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:17.779093 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.778957    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7mkt\" (UniqueName: \"kubernetes.io/projected/2f527a21-a030-49c3-86f8-dd6407bc3c23-kube-api-access-k7mkt\") pod \"sub-test-model-8fdd5b-kserve-6b965b5749-thzfq\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:17.779093 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.778990    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-model-cache\") pod \"sub-test-model-8fdd5b-kserve-6b965b5749-thzfq\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:17.779093 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.779060    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-dshm\") pod \"sub-test-model-8fdd5b-kserve-6b965b5749-thzfq\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:17.779392 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.779318    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-home\") pod \"sub-test-model-8fdd5b-kserve-6b965b5749-thzfq\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:17.779392 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.779377    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-tmp-dir\") pod \"sub-test-model-8fdd5b-kserve-6b965b5749-thzfq\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:17.779507 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.779480    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-model-cache\") pod \"sub-test-model-8fdd5b-kserve-6b965b5749-thzfq\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:17.781272 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.781254    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-dshm\") pod \"sub-test-model-8fdd5b-kserve-6b965b5749-thzfq\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:17.781600 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.781584    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f527a21-a030-49c3-86f8-dd6407bc3c23-tls-certs\") pod \"sub-test-model-8fdd5b-kserve-6b965b5749-thzfq\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:17.786491 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.786471    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7mkt\" (UniqueName: \"kubernetes.io/projected/2f527a21-a030-49c3-86f8-dd6407bc3c23-kube-api-access-k7mkt\") pod \"sub-test-model-8fdd5b-kserve-6b965b5749-thzfq\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:17.890267 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:17.890168    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:18.159180 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:18.159150    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"]
Sep 18 06:56:18.161030 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:56:18.161005    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f527a21_a030_49c3_86f8_dd6407bc3c23.slice/crio-788802aafa67dc03ded1ee86a3dff099efffe679fba4cfe1a6347dd4ca0708a3 WatchSource:0}: Error finding container 788802aafa67dc03ded1ee86a3dff099efffe679fba4cfe1a6347dd4ca0708a3: Status 404 returned error can't find the container with id 788802aafa67dc03ded1ee86a3dff099efffe679fba4cfe1a6347dd4ca0708a3
Sep 18 06:56:18.781712 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:18.781667    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq" event={"ID":"2f527a21-a030-49c3-86f8-dd6407bc3c23","Type":"ContainerStarted","Data":"e67d4d786eee5ae42ca6ceed0f9ec0da2d006247202c15ef223d8d8f4f0929f8"}
Sep 18 06:56:18.782152 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:18.781723    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq" event={"ID":"2f527a21-a030-49c3-86f8-dd6407bc3c23","Type":"ContainerStarted","Data":"788802aafa67dc03ded1ee86a3dff099efffe679fba4cfe1a6347dd4ca0708a3"}
Sep 18 06:56:18.796836 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:18.796792    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq" podStartSLOduration=1.796774937 podStartE2EDuration="1.796774937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:56:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:56:18.795715008 +0000 UTC m=+2044.477242752" watchObservedRunningTime="2026-09-18 06:56:18.796774937 +0000 UTC m=+2044.478302682"
Sep 18 06:56:21.586015 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:21.585976    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:56:27.891128 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:27.891047    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:27.891128 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:27.891100    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:27.903607 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:27.903584    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:28.853018 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:28.852983    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:56:34.629511 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:34.629476    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:56:34.730536 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:34.730503    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:56:35.455299 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:35.455267    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:56:37.187884 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:37.187848    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:56:50.482239 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:50.482186    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:56:51.628680 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:51.628648    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:56:54.083891 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:54.083858    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7756-806e0a"]
Sep 18 06:56:54.087438 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:54.087421    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7756-806e0a"
Sep 18 06:56:54.089787 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:54.089760    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-h78xn\""
Sep 18 06:56:54.092497 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:54.092473    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7756-806e0a"]
Sep 18 06:56:54.132433 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:54.132406    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzts2\" (UniqueName: \"kubernetes.io/projected/263d647b-0928-4cf7-9746-8f88f66011c3-kube-api-access-mzts2\") pod \"test-curl-7756-806e0a\" (UID: \"263d647b-0928-4cf7-9746-8f88f66011c3\") " pod="opendatahub/test-curl-7756-806e0a"
Sep 18 06:56:54.233555 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:54.233523    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzts2\" (UniqueName: \"kubernetes.io/projected/263d647b-0928-4cf7-9746-8f88f66011c3-kube-api-access-mzts2\") pod \"test-curl-7756-806e0a\" (UID: \"263d647b-0928-4cf7-9746-8f88f66011c3\") " pod="opendatahub/test-curl-7756-806e0a"
Sep 18 06:56:54.240972 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:54.240945    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzts2\" (UniqueName: \"kubernetes.io/projected/263d647b-0928-4cf7-9746-8f88f66011c3-kube-api-access-mzts2\") pod \"test-curl-7756-806e0a\" (UID: \"263d647b-0928-4cf7-9746-8f88f66011c3\") " pod="opendatahub/test-curl-7756-806e0a"
Sep 18 06:56:54.398813 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:54.398786    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7756-806e0a"
Sep 18 06:56:54.664553 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:54.664523    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7756-806e0a"]
Sep 18 06:56:54.666468 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:56:54.666439    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod263d647b_0928_4cf7_9746_8f88f66011c3.slice/crio-ecfa5cb964b48787fd14d24c59ef35f67b0e90fab193a3017becb144bc82b9d0 WatchSource:0}: Error finding container ecfa5cb964b48787fd14d24c59ef35f67b0e90fab193a3017becb144bc82b9d0: Status 404 returned error can't find the container with id ecfa5cb964b48787fd14d24c59ef35f67b0e90fab193a3017becb144bc82b9d0
Sep 18 06:56:54.973507 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:54.973409    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7756-806e0a" event={"ID":"263d647b-0928-4cf7-9746-8f88f66011c3","Type":"ContainerStarted","Data":"c9caf91c8ca28c679071134c79ebccb2622863051293027cc3f75b59a3911bfa"}
Sep 18 06:56:54.973507 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:54.973449    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7756-806e0a" event={"ID":"263d647b-0928-4cf7-9746-8f88f66011c3","Type":"ContainerStarted","Data":"ecfa5cb964b48787fd14d24c59ef35f67b0e90fab193a3017becb144bc82b9d0"}
Sep 18 06:56:54.987278 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:54.987238    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7756-806e0a" podStartSLOduration=0.987208657 podStartE2EDuration="987.208657ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:56:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:56:54.985390497 +0000 UTC m=+2080.666918242" watchObservedRunningTime="2026-09-18 06:56:54.987208657 +0000 UTC m=+2080.668736400"
Sep 18 06:56:55.529375 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:55.529338    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7756-806e0a"]
Sep 18 06:56:55.533255 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:55.533213    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7756-806e0a"]
Sep 18 06:56:55.660305 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:55.660275    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7756-9e60d1"]
Sep 18 06:56:55.660692 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:55.660678    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="263d647b-0928-4cf7-9746-8f88f66011c3" containerName="test-curl-7756-806e0a"
Sep 18 06:56:55.660740 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:55.660693    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="263d647b-0928-4cf7-9746-8f88f66011c3" containerName="test-curl-7756-806e0a"
Sep 18 06:56:55.660776 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:55.660771    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="263d647b-0928-4cf7-9746-8f88f66011c3" containerName="test-curl-7756-806e0a"
Sep 18 06:56:55.663922 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:55.663906    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7756-9e60d1"
Sep 18 06:56:55.669178 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:55.669154    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7756-9e60d1"]
Sep 18 06:56:55.750189 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:55.750157    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68k4z\" (UniqueName: \"kubernetes.io/projected/4e262f4c-f6d4-4265-8939-4b4d7095382a-kube-api-access-68k4z\") pod \"test-curl-7756-9e60d1\" (UID: \"4e262f4c-f6d4-4265-8939-4b4d7095382a\") " pod="opendatahub/test-curl-7756-9e60d1"
Sep 18 06:56:55.850984 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:55.850951    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68k4z\" (UniqueName: \"kubernetes.io/projected/4e262f4c-f6d4-4265-8939-4b4d7095382a-kube-api-access-68k4z\") pod \"test-curl-7756-9e60d1\" (UID: \"4e262f4c-f6d4-4265-8939-4b4d7095382a\") " pod="opendatahub/test-curl-7756-9e60d1"
Sep 18 06:56:55.858723 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:55.858701    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-68k4z\" (UniqueName: \"kubernetes.io/projected/4e262f4c-f6d4-4265-8939-4b4d7095382a-kube-api-access-68k4z\") pod \"test-curl-7756-9e60d1\" (UID: \"4e262f4c-f6d4-4265-8939-4b4d7095382a\") " pod="opendatahub/test-curl-7756-9e60d1"
Sep 18 06:56:55.975408 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:55.975334    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7756-9e60d1"
Sep 18 06:56:55.978932 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:56:55.978913    2568 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7756-806e0a" container="test-curl-7756-806e0a"
Sep 18 06:56:56.240367 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:56.240340    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7756-9e60d1"]
Sep 18 06:56:56.242091 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:56:56.242066    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e262f4c_f6d4_4265_8939_4b4d7095382a.slice/crio-095ada23ca2a23c440e07311b6eea5ea3f2f0897166b868ab859476f83f6ea0a WatchSource:0}: Error finding container 095ada23ca2a23c440e07311b6eea5ea3f2f0897166b868ab859476f83f6ea0a: Status 404 returned error can't find the container with id 095ada23ca2a23c440e07311b6eea5ea3f2f0897166b868ab859476f83f6ea0a
Sep 18 06:56:56.985332 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:56.985291    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7756-9e60d1" event={"ID":"4e262f4c-f6d4-4265-8939-4b4d7095382a","Type":"ContainerStarted","Data":"0f2431f8f68adeed535e938aec219ee70219782a2b319b52617bda4a62f39b0f"}
Sep 18 06:56:56.985332 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:56.985341    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7756-9e60d1" event={"ID":"4e262f4c-f6d4-4265-8939-4b4d7095382a","Type":"ContainerStarted","Data":"095ada23ca2a23c440e07311b6eea5ea3f2f0897166b868ab859476f83f6ea0a"}
Sep 18 06:56:56.985848 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:56.985406    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7756-806e0a" podUID="263d647b-0928-4cf7-9746-8f88f66011c3" containerName="test-curl-7756-806e0a" containerID="cri-o://c9caf91c8ca28c679071134c79ebccb2622863051293027cc3f75b59a3911bfa" gracePeriod=2
Sep 18 06:56:56.997907 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:56.997862    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7756-9e60d1" podStartSLOduration=1.9978467850000001 podStartE2EDuration="1.997846785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:56:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:56:56.997296813 +0000 UTC m=+2082.678824556" watchObservedRunningTime="2026-09-18 06:56:56.997846785 +0000 UTC m=+2082.679374529"
Sep 18 06:56:57.547037 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:57.547008    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7756-9e60d1"]
Sep 18 06:56:57.550159 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:57.550131    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7756-9e60d1"]
Sep 18 06:56:57.990546 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:56:57.990509    2568 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7756-9e60d1" container="test-curl-7756-9e60d1"
Sep 18 06:56:58.996401 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:58.996363    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7756-9e60d1" podUID="4e262f4c-f6d4-4265-8939-4b4d7095382a" containerName="test-curl-7756-9e60d1" containerID="cri-o://0f2431f8f68adeed535e938aec219ee70219782a2b319b52617bda4a62f39b0f" gracePeriod=2
Sep 18 06:56:59.136615 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:59.136593    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7756-806e0a"
Sep 18 06:56:59.183956 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:59.183927    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzts2\" (UniqueName: \"kubernetes.io/projected/263d647b-0928-4cf7-9746-8f88f66011c3-kube-api-access-mzts2\") pod \"263d647b-0928-4cf7-9746-8f88f66011c3\" (UID: \"263d647b-0928-4cf7-9746-8f88f66011c3\") "
Sep 18 06:56:59.185975 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:59.185946    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/263d647b-0928-4cf7-9746-8f88f66011c3-kube-api-access-mzts2" (OuterVolumeSpecName: "kube-api-access-mzts2") pod "263d647b-0928-4cf7-9746-8f88f66011c3" (UID: "263d647b-0928-4cf7-9746-8f88f66011c3"). InnerVolumeSpecName "kube-api-access-mzts2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:56:59.285724 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:59.285646    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzts2\" (UniqueName: \"kubernetes.io/projected/263d647b-0928-4cf7-9746-8f88f66011c3-kube-api-access-mzts2\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:56:59.479876 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:59.479841    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"]
Sep 18 06:56:59.498465 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:56:59.498434    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:57:00.002780 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:00.002746    2568 generic.go:356] "Generic (PLEG): container finished" podID="263d647b-0928-4cf7-9746-8f88f66011c3" containerID="c9caf91c8ca28c679071134c79ebccb2622863051293027cc3f75b59a3911bfa" exitCode=137
Sep 18 06:57:00.003251 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:00.002789    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7756-806e0a"
Sep 18 06:57:00.003251 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:00.002847    2568 scope.go:117] "RemoveContainer" containerID="c9caf91c8ca28c679071134c79ebccb2622863051293027cc3f75b59a3911bfa"
Sep 18 06:57:00.012344 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:00.012326    2568 scope.go:117] "RemoveContainer" containerID="c9caf91c8ca28c679071134c79ebccb2622863051293027cc3f75b59a3911bfa"
Sep 18 06:57:00.012669 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:57:00.012650    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9caf91c8ca28c679071134c79ebccb2622863051293027cc3f75b59a3911bfa\": container with ID starting with c9caf91c8ca28c679071134c79ebccb2622863051293027cc3f75b59a3911bfa not found: ID does not exist" containerID="c9caf91c8ca28c679071134c79ebccb2622863051293027cc3f75b59a3911bfa"
Sep 18 06:57:00.012776 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:00.012675    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9caf91c8ca28c679071134c79ebccb2622863051293027cc3f75b59a3911bfa"} err="failed to get container status \"c9caf91c8ca28c679071134c79ebccb2622863051293027cc3f75b59a3911bfa\": rpc error: code = NotFound desc = could not find container \"c9caf91c8ca28c679071134c79ebccb2622863051293027cc3f75b59a3911bfa\": container with ID starting with c9caf91c8ca28c679071134c79ebccb2622863051293027cc3f75b59a3911bfa not found: ID does not exist"
Sep 18 06:57:00.013807 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:57:00.013782    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7756-806e0a\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="263d647b-0928-4cf7-9746-8f88f66011c3" pod="opendatahub/test-curl-7756-806e0a"
Sep 18 06:57:00.470891 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:00.470851    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"]
Sep 18 06:57:00.473198 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:57:00.473157    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7756-806e0a\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="263d647b-0928-4cf7-9746-8f88f66011c3" pod="opendatahub/test-curl-7756-806e0a"
Sep 18 06:57:00.484793 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:00.484769    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:57:00.549286 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:00.549254    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="263d647b-0928-4cf7-9746-8f88f66011c3" path="/var/lib/kubelet/pods/263d647b-0928-4cf7-9746-8f88f66011c3/volumes"
Sep 18 06:57:01.170438 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:01.170415    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7756-9e60d1"
Sep 18 06:57:01.305990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:01.305906    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-68k4z\" (UniqueName: \"kubernetes.io/projected/4e262f4c-f6d4-4265-8939-4b4d7095382a-kube-api-access-68k4z\") pod \"4e262f4c-f6d4-4265-8939-4b4d7095382a\" (UID: \"4e262f4c-f6d4-4265-8939-4b4d7095382a\") "
Sep 18 06:57:01.308050 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:01.308024    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e262f4c-f6d4-4265-8939-4b4d7095382a-kube-api-access-68k4z" (OuterVolumeSpecName: "kube-api-access-68k4z") pod "4e262f4c-f6d4-4265-8939-4b4d7095382a" (UID: "4e262f4c-f6d4-4265-8939-4b4d7095382a"). InnerVolumeSpecName "kube-api-access-68k4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:57:01.406939 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:01.406909    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68k4z\" (UniqueName: \"kubernetes.io/projected/4e262f4c-f6d4-4265-8939-4b4d7095382a-kube-api-access-68k4z\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:57:02.015759 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:02.015725    2568 generic.go:356] "Generic (PLEG): container finished" podID="4e262f4c-f6d4-4265-8939-4b4d7095382a" containerID="0f2431f8f68adeed535e938aec219ee70219782a2b319b52617bda4a62f39b0f" exitCode=137
Sep 18 06:57:02.015944 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:02.015769    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7756-9e60d1"
Sep 18 06:57:02.015944 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:02.015823    2568 scope.go:117] "RemoveContainer" containerID="0f2431f8f68adeed535e938aec219ee70219782a2b319b52617bda4a62f39b0f"
Sep 18 06:57:02.024437 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:02.024418    2568 scope.go:117] "RemoveContainer" containerID="0f2431f8f68adeed535e938aec219ee70219782a2b319b52617bda4a62f39b0f"
Sep 18 06:57:02.024740 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:57:02.024721    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f2431f8f68adeed535e938aec219ee70219782a2b319b52617bda4a62f39b0f\": container with ID starting with 0f2431f8f68adeed535e938aec219ee70219782a2b319b52617bda4a62f39b0f not found: ID does not exist" containerID="0f2431f8f68adeed535e938aec219ee70219782a2b319b52617bda4a62f39b0f"
Sep 18 06:57:02.024794 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:02.024751    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f2431f8f68adeed535e938aec219ee70219782a2b319b52617bda4a62f39b0f"} err="failed to get container status \"0f2431f8f68adeed535e938aec219ee70219782a2b319b52617bda4a62f39b0f\": rpc error: code = NotFound desc = could not find container \"0f2431f8f68adeed535e938aec219ee70219782a2b319b52617bda4a62f39b0f\": container with ID starting with 0f2431f8f68adeed535e938aec219ee70219782a2b319b52617bda4a62f39b0f not found: ID does not exist"
Sep 18 06:57:02.026136 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:57:02.026112    2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7756-9e60d1\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="4e262f4c-f6d4-4265-8939-4b4d7095382a" pod="opendatahub/test-curl-7756-9e60d1"
Sep 18 06:57:02.543486 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:02.543456    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e262f4c-f6d4-4265-8939-4b4d7095382a" path="/var/lib/kubelet/pods/4e262f4c-f6d4-4265-8939-4b4d7095382a/volumes"
Sep 18 06:57:03.594802 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:03.594768    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:57:14.499755 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:14.499710    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng" podUID="9ff07573-2b58-456e-85c7-79974d9a5c94" containerName="main" containerID="cri-o://0e3e3fe3c61e4c8d4c0d2b8afc7ea328282c7fc5a003f39d6b57fa7986474195" gracePeriod=45
Sep 18 06:57:14.752608 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:14.752552    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:57:14.938725 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:14.938693    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpxbw\" (UniqueName: \"kubernetes.io/projected/9ff07573-2b58-456e-85c7-79974d9a5c94-kube-api-access-qpxbw\") pod \"9ff07573-2b58-456e-85c7-79974d9a5c94\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") "
Sep 18 06:57:14.938902 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:14.938749    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-home\") pod \"9ff07573-2b58-456e-85c7-79974d9a5c94\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") "
Sep 18 06:57:14.938902 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:14.938785    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ff07573-2b58-456e-85c7-79974d9a5c94-tls-certs\") pod \"9ff07573-2b58-456e-85c7-79974d9a5c94\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") "
Sep 18 06:57:14.938902 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:14.938813    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-tmp-dir\") pod \"9ff07573-2b58-456e-85c7-79974d9a5c94\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") "
Sep 18 06:57:14.938902 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:14.938855    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-model-cache\") pod \"9ff07573-2b58-456e-85c7-79974d9a5c94\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") "
Sep 18 06:57:14.939100 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:14.938923    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-dshm\") pod \"9ff07573-2b58-456e-85c7-79974d9a5c94\" (UID: \"9ff07573-2b58-456e-85c7-79974d9a5c94\") "
Sep 18 06:57:14.939160 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:14.939090    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-home" (OuterVolumeSpecName: "home") pod "9ff07573-2b58-456e-85c7-79974d9a5c94" (UID: "9ff07573-2b58-456e-85c7-79974d9a5c94"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:57:14.939160 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:14.939105    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9ff07573-2b58-456e-85c7-79974d9a5c94" (UID: "9ff07573-2b58-456e-85c7-79974d9a5c94"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:57:14.939308 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:14.939283    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-model-cache" (OuterVolumeSpecName: "model-cache") pod "9ff07573-2b58-456e-85c7-79974d9a5c94" (UID: "9ff07573-2b58-456e-85c7-79974d9a5c94"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:57:14.939401 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:14.939377    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:57:14.939401 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:14.939403    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:57:14.939591 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:14.939415    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:57:14.941181 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:14.941163    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-dshm" (OuterVolumeSpecName: "dshm") pod "9ff07573-2b58-456e-85c7-79974d9a5c94" (UID: "9ff07573-2b58-456e-85c7-79974d9a5c94"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:57:14.941389 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:14.941371    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ff07573-2b58-456e-85c7-79974d9a5c94-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9ff07573-2b58-456e-85c7-79974d9a5c94" (UID: "9ff07573-2b58-456e-85c7-79974d9a5c94"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:57:14.941472 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:14.941402    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ff07573-2b58-456e-85c7-79974d9a5c94-kube-api-access-qpxbw" (OuterVolumeSpecName: "kube-api-access-qpxbw") pod "9ff07573-2b58-456e-85c7-79974d9a5c94" (UID: "9ff07573-2b58-456e-85c7-79974d9a5c94"). InnerVolumeSpecName "kube-api-access-qpxbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:57:15.040285 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.040195    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ff07573-2b58-456e-85c7-79974d9a5c94-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:57:15.040285 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.040249    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ff07573-2b58-456e-85c7-79974d9a5c94-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:57:15.040285 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.040262    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpxbw\" (UniqueName: \"kubernetes.io/projected/9ff07573-2b58-456e-85c7-79974d9a5c94-kube-api-access-qpxbw\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:57:15.083340 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.083307    2568 generic.go:356] "Generic (PLEG): container finished" podID="9ff07573-2b58-456e-85c7-79974d9a5c94" containerID="0e3e3fe3c61e4c8d4c0d2b8afc7ea328282c7fc5a003f39d6b57fa7986474195" exitCode=0
Sep 18 06:57:15.083474 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.083351    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"
Sep 18 06:57:15.083474 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.083372    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng" event={"ID":"9ff07573-2b58-456e-85c7-79974d9a5c94","Type":"ContainerDied","Data":"0e3e3fe3c61e4c8d4c0d2b8afc7ea328282c7fc5a003f39d6b57fa7986474195"}
Sep 18 06:57:15.083474 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.083410    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng" event={"ID":"9ff07573-2b58-456e-85c7-79974d9a5c94","Type":"ContainerDied","Data":"6ffd3c48836e5b63e794e76204e51598470f01dd21d0d21b2d4ccf475ae44418"}
Sep 18 06:57:15.083474 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.083432    2568 scope.go:117] "RemoveContainer" containerID="0e3e3fe3c61e4c8d4c0d2b8afc7ea328282c7fc5a003f39d6b57fa7986474195"
Sep 18 06:57:15.092691 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.092673    2568 scope.go:117] "RemoveContainer" containerID="0e3e3fe3c61e4c8d4c0d2b8afc7ea328282c7fc5a003f39d6b57fa7986474195"
Sep 18 06:57:15.092950 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:57:15.092934    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e3e3fe3c61e4c8d4c0d2b8afc7ea328282c7fc5a003f39d6b57fa7986474195\": container with ID starting with 0e3e3fe3c61e4c8d4c0d2b8afc7ea328282c7fc5a003f39d6b57fa7986474195 not found: ID does not exist" containerID="0e3e3fe3c61e4c8d4c0d2b8afc7ea328282c7fc5a003f39d6b57fa7986474195"
Sep 18 06:57:15.092992 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.092958    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e3e3fe3c61e4c8d4c0d2b8afc7ea328282c7fc5a003f39d6b57fa7986474195"} err="failed to get container status \"0e3e3fe3c61e4c8d4c0d2b8afc7ea328282c7fc5a003f39d6b57fa7986474195\": rpc error: code = NotFound desc = could not find container \"0e3e3fe3c61e4c8d4c0d2b8afc7ea328282c7fc5a003f39d6b57fa7986474195\": container with ID starting with 0e3e3fe3c61e4c8d4c0d2b8afc7ea328282c7fc5a003f39d6b57fa7986474195 not found: ID does not exist"
Sep 18 06:57:15.103528 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.103508    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"]
Sep 18 06:57:15.107126 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.107104    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ec2077/sub-test-model-ec2077-kserve-6757d78567-7qlng"]
Sep 18 06:57:15.490626 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.490584    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq" podUID="2f527a21-a030-49c3-86f8-dd6407bc3c23" containerName="main" containerID="cri-o://e67d4d786eee5ae42ca6ceed0f9ec0da2d006247202c15ef223d8d8f4f0929f8" gracePeriod=45
Sep 18 06:57:15.745177 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.745117    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:57:15.848143 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.848112    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-tmp-dir\") pod \"2f527a21-a030-49c3-86f8-dd6407bc3c23\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") "
Sep 18 06:57:15.848345 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.848187    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-home\") pod \"2f527a21-a030-49c3-86f8-dd6407bc3c23\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") "
Sep 18 06:57:15.848345 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.848233    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-dshm\") pod \"2f527a21-a030-49c3-86f8-dd6407bc3c23\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") "
Sep 18 06:57:15.848345 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.848260    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-model-cache\") pod \"2f527a21-a030-49c3-86f8-dd6407bc3c23\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") "
Sep 18 06:57:15.848345 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.848327    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7mkt\" (UniqueName: \"kubernetes.io/projected/2f527a21-a030-49c3-86f8-dd6407bc3c23-kube-api-access-k7mkt\") pod \"2f527a21-a030-49c3-86f8-dd6407bc3c23\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") "
Sep 18 06:57:15.848604 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.848431    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2f527a21-a030-49c3-86f8-dd6407bc3c23" (UID: "2f527a21-a030-49c3-86f8-dd6407bc3c23"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:57:15.848604 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.848452    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f527a21-a030-49c3-86f8-dd6407bc3c23-tls-certs\") pod \"2f527a21-a030-49c3-86f8-dd6407bc3c23\" (UID: \"2f527a21-a030-49c3-86f8-dd6407bc3c23\") "
Sep 18 06:57:15.848604 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.848482    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-home" (OuterVolumeSpecName: "home") pod "2f527a21-a030-49c3-86f8-dd6407bc3c23" (UID: "2f527a21-a030-49c3-86f8-dd6407bc3c23"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:57:15.848773 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.848600    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-model-cache" (OuterVolumeSpecName: "model-cache") pod "2f527a21-a030-49c3-86f8-dd6407bc3c23" (UID: "2f527a21-a030-49c3-86f8-dd6407bc3c23"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:57:15.848942 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.848866    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:57:15.848942 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.848890    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:57:15.848942 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.848903    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:57:15.850505 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.850469    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-dshm" (OuterVolumeSpecName: "dshm") pod "2f527a21-a030-49c3-86f8-dd6407bc3c23" (UID: "2f527a21-a030-49c3-86f8-dd6407bc3c23"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:57:15.850617 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.850534    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f527a21-a030-49c3-86f8-dd6407bc3c23-kube-api-access-k7mkt" (OuterVolumeSpecName: "kube-api-access-k7mkt") pod "2f527a21-a030-49c3-86f8-dd6407bc3c23" (UID: "2f527a21-a030-49c3-86f8-dd6407bc3c23"). InnerVolumeSpecName "kube-api-access-k7mkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:57:15.850617 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.850558    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f527a21-a030-49c3-86f8-dd6407bc3c23-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2f527a21-a030-49c3-86f8-dd6407bc3c23" (UID: "2f527a21-a030-49c3-86f8-dd6407bc3c23"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:57:15.950321 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.950290    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f527a21-a030-49c3-86f8-dd6407bc3c23-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:57:15.950321 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.950319    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7mkt\" (UniqueName: \"kubernetes.io/projected/2f527a21-a030-49c3-86f8-dd6407bc3c23-kube-api-access-k7mkt\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:57:15.950511 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:15.950331    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f527a21-a030-49c3-86f8-dd6407bc3c23-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:57:16.089985 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:16.089894    2568 generic.go:356] "Generic (PLEG): container finished" podID="2f527a21-a030-49c3-86f8-dd6407bc3c23" containerID="e67d4d786eee5ae42ca6ceed0f9ec0da2d006247202c15ef223d8d8f4f0929f8" exitCode=0
Sep 18 06:57:16.089985 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:16.089937    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"
Sep 18 06:57:16.089985 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:16.089975    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq" event={"ID":"2f527a21-a030-49c3-86f8-dd6407bc3c23","Type":"ContainerDied","Data":"e67d4d786eee5ae42ca6ceed0f9ec0da2d006247202c15ef223d8d8f4f0929f8"}
Sep 18 06:57:16.090300 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:16.090019    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq" event={"ID":"2f527a21-a030-49c3-86f8-dd6407bc3c23","Type":"ContainerDied","Data":"788802aafa67dc03ded1ee86a3dff099efffe679fba4cfe1a6347dd4ca0708a3"}
Sep 18 06:57:16.090300 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:16.090042    2568 scope.go:117] "RemoveContainer" containerID="e67d4d786eee5ae42ca6ceed0f9ec0da2d006247202c15ef223d8d8f4f0929f8"
Sep 18 06:57:16.099006 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:16.098991    2568 scope.go:117] "RemoveContainer" containerID="e67d4d786eee5ae42ca6ceed0f9ec0da2d006247202c15ef223d8d8f4f0929f8"
Sep 18 06:57:16.099273 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:57:16.099253    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e67d4d786eee5ae42ca6ceed0f9ec0da2d006247202c15ef223d8d8f4f0929f8\": container with ID starting with e67d4d786eee5ae42ca6ceed0f9ec0da2d006247202c15ef223d8d8f4f0929f8 not found: ID does not exist" containerID="e67d4d786eee5ae42ca6ceed0f9ec0da2d006247202c15ef223d8d8f4f0929f8"
Sep 18 06:57:16.099349 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:16.099281    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e67d4d786eee5ae42ca6ceed0f9ec0da2d006247202c15ef223d8d8f4f0929f8"} err="failed to get container status \"e67d4d786eee5ae42ca6ceed0f9ec0da2d006247202c15ef223d8d8f4f0929f8\": rpc error: code = NotFound desc = could not find container \"e67d4d786eee5ae42ca6ceed0f9ec0da2d006247202c15ef223d8d8f4f0929f8\": container with ID starting with e67d4d786eee5ae42ca6ceed0f9ec0da2d006247202c15ef223d8d8f4f0929f8 not found: ID does not exist"
Sep 18 06:57:16.109518 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:16.109496    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"]
Sep 18 06:57:16.112468 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:16.112447    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8fdd5b/sub-test-model-8fdd5b-kserve-6b965b5749-thzfq"]
Sep 18 06:57:16.544083 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:16.544054    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f527a21-a030-49c3-86f8-dd6407bc3c23" path="/var/lib/kubelet/pods/2f527a21-a030-49c3-86f8-dd6407bc3c23/volumes"
Sep 18 06:57:16.544480 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:16.544466    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ff07573-2b58-456e-85c7-79974d9a5c94" path="/var/lib/kubelet/pods/9ff07573-2b58-456e-85c7-79974d9a5c94/volumes"
Sep 18 06:57:28.991464 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:28.991432    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"]
Sep 18 06:57:28.991851 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:28.991807    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e262f4c-f6d4-4265-8939-4b4d7095382a" containerName="test-curl-7756-9e60d1"
Sep 18 06:57:28.991851 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:28.991818    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e262f4c-f6d4-4265-8939-4b4d7095382a" containerName="test-curl-7756-9e60d1"
Sep 18 06:57:28.991851 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:28.991830    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f527a21-a030-49c3-86f8-dd6407bc3c23" containerName="main"
Sep 18 06:57:28.991851 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:28.991838    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f527a21-a030-49c3-86f8-dd6407bc3c23" containerName="main"
Sep 18 06:57:28.992171 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:28.991854    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ff07573-2b58-456e-85c7-79974d9a5c94" containerName="main"
Sep 18 06:57:28.992171 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:28.991863    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ff07573-2b58-456e-85c7-79974d9a5c94" containerName="main"
Sep 18 06:57:28.992171 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:28.991972    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e262f4c-f6d4-4265-8939-4b4d7095382a" containerName="test-curl-7756-9e60d1"
Sep 18 06:57:28.992171 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:28.991989    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ff07573-2b58-456e-85c7-79974d9a5c94" containerName="main"
Sep 18 06:57:28.992171 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:28.992002    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f527a21-a030-49c3-86f8-dd6407bc3c23" containerName="main"
Sep 18 06:57:28.996939 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:28.996920    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:28.999302 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:28.999273    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ec2077\"/\"default-dockercfg-b954k\""
Sep 18 06:57:29.000329 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.000309    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ec2077\"/\"rate-test-model-ec2077-kserve-self-signed-certs\""
Sep 18 06:57:29.000446 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.000334    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ec2077\"/\"openshift-service-ca.crt\""
Sep 18 06:57:29.000446 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.000336    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ec2077\"/\"kube-root-ca.crt\""
Sep 18 06:57:29.003509 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.003486    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"]
Sep 18 06:57:29.016687 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.016665    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:57:29.077732 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.077695    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-tls-certs\") pod \"rate-test-model-ec2077-kserve-84647cd97f-s9xnd\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:29.077894 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.077761    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-dshm\") pod \"rate-test-model-ec2077-kserve-84647cd97f-s9xnd\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:29.077894 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.077791    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-model-cache\") pod \"rate-test-model-ec2077-kserve-84647cd97f-s9xnd\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:29.077894 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.077816    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh6cd\" (UniqueName: \"kubernetes.io/projected/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-kube-api-access-sh6cd\") pod \"rate-test-model-ec2077-kserve-84647cd97f-s9xnd\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:29.077894 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.077872    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-home\") pod \"rate-test-model-ec2077-kserve-84647cd97f-s9xnd\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:29.078037 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.077977    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-tmp-dir\") pod \"rate-test-model-ec2077-kserve-84647cd97f-s9xnd\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:29.179392 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.179355    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-tmp-dir\") pod \"rate-test-model-ec2077-kserve-84647cd97f-s9xnd\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:29.179599 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.179435    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-tls-certs\") pod \"rate-test-model-ec2077-kserve-84647cd97f-s9xnd\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:29.179599 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.179474    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-dshm\") pod \"rate-test-model-ec2077-kserve-84647cd97f-s9xnd\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:29.179599 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.179498    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-model-cache\") pod \"rate-test-model-ec2077-kserve-84647cd97f-s9xnd\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:29.179599 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.179523    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh6cd\" (UniqueName: \"kubernetes.io/projected/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-kube-api-access-sh6cd\") pod \"rate-test-model-ec2077-kserve-84647cd97f-s9xnd\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:29.179599 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.179568    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-home\") pod \"rate-test-model-ec2077-kserve-84647cd97f-s9xnd\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:29.179872 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.179812    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-tmp-dir\") pod \"rate-test-model-ec2077-kserve-84647cd97f-s9xnd\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:29.179872 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.179835    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-model-cache\") pod \"rate-test-model-ec2077-kserve-84647cd97f-s9xnd\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:29.179953 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.179913    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-home\") pod \"rate-test-model-ec2077-kserve-84647cd97f-s9xnd\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:29.181715 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.181690    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-dshm\") pod \"rate-test-model-ec2077-kserve-84647cd97f-s9xnd\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:29.181990 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.181974    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-tls-certs\") pod \"rate-test-model-ec2077-kserve-84647cd97f-s9xnd\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:29.188017 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.187992    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh6cd\" (UniqueName: \"kubernetes.io/projected/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-kube-api-access-sh6cd\") pod \"rate-test-model-ec2077-kserve-84647cd97f-s9xnd\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") " pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:29.307584 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.307487    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:29.578244 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:29.578205    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"]
Sep 18 06:57:29.579582 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:57:29.579555    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod11c5e885_4a38_4a83_90d7_6ceeeafff9d1.slice/crio-55bdddbb82a631ecf80f11e6ef2d5d722089b70705e20cb21d8016cf085368b5 WatchSource:0}: Error finding container 55bdddbb82a631ecf80f11e6ef2d5d722089b70705e20cb21d8016cf085368b5: Status 404 returned error can't find the container with id 55bdddbb82a631ecf80f11e6ef2d5d722089b70705e20cb21d8016cf085368b5
Sep 18 06:57:30.165016 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:30.164981    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd" event={"ID":"11c5e885-4a38-4a83-90d7-6ceeeafff9d1","Type":"ContainerStarted","Data":"a1c317f30a04a626836159624f2f493e9b2c7370636621951df47e3f4f38835a"}
Sep 18 06:57:30.165424 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:30.165024    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd" event={"ID":"11c5e885-4a38-4a83-90d7-6ceeeafff9d1","Type":"ContainerStarted","Data":"55bdddbb82a631ecf80f11e6ef2d5d722089b70705e20cb21d8016cf085368b5"}
Sep 18 06:57:30.180300 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:30.180254    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd" podStartSLOduration=2.180238227 podStartE2EDuration="2.180238227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:57:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:57:30.180160498 +0000 UTC m=+2115.861688241" watchObservedRunningTime="2026-09-18 06:57:30.180238227 +0000 UTC m=+2115.861765968"
Sep 18 06:57:32.386662 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:32.386628    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:57:39.308553 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:39.308517    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:39.308934 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:39.308575    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:39.321133 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:39.321112    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:40.227045 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:40.227017    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:57:48.694537 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.694501    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"]
Sep 18 06:57:48.698022 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.698001    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:48.700209 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.700188    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-8fdd5b\"/\"default-dockercfg-nl8gc\""
Sep 18 06:57:48.701325 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.701306    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-8fdd5b\"/\"rate-test-model-8fdd5b-kserve-self-signed-certs\""
Sep 18 06:57:48.701458 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.701325    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-8fdd5b\"/\"openshift-service-ca.crt\""
Sep 18 06:57:48.701458 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.701333    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-8fdd5b\"/\"kube-root-ca.crt\""
Sep 18 06:57:48.706610 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.706588    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"]
Sep 18 06:57:48.715320 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.715296    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:57:48.867514 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.867478    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-tmp-dir\") pod \"rate-test-model-8fdd5b-kserve-7554bcc75-pllvl\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:48.867719 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.867626    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-dshm\") pod \"rate-test-model-8fdd5b-kserve-7554bcc75-pllvl\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:48.867719 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.867699    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfcpx\" (UniqueName: \"kubernetes.io/projected/897e64e3-fb7c-4a02-9c6f-1012b2810a53-kube-api-access-pfcpx\") pod \"rate-test-model-8fdd5b-kserve-7554bcc75-pllvl\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:48.867857 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.867766    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-home\") pod \"rate-test-model-8fdd5b-kserve-7554bcc75-pllvl\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:48.867915 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.867893    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-model-cache\") pod \"rate-test-model-8fdd5b-kserve-7554bcc75-pllvl\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:48.867980 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.867959    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/897e64e3-fb7c-4a02-9c6f-1012b2810a53-tls-certs\") pod \"rate-test-model-8fdd5b-kserve-7554bcc75-pllvl\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:48.969530 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.969448    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-tmp-dir\") pod \"rate-test-model-8fdd5b-kserve-7554bcc75-pllvl\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:48.969530 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.969493    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-dshm\") pod \"rate-test-model-8fdd5b-kserve-7554bcc75-pllvl\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:48.969530 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.969511    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfcpx\" (UniqueName: \"kubernetes.io/projected/897e64e3-fb7c-4a02-9c6f-1012b2810a53-kube-api-access-pfcpx\") pod \"rate-test-model-8fdd5b-kserve-7554bcc75-pllvl\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:48.969530 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.969533    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-home\") pod \"rate-test-model-8fdd5b-kserve-7554bcc75-pllvl\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:48.969876 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.969593    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-model-cache\") pod \"rate-test-model-8fdd5b-kserve-7554bcc75-pllvl\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:48.969876 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.969633    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/897e64e3-fb7c-4a02-9c6f-1012b2810a53-tls-certs\") pod \"rate-test-model-8fdd5b-kserve-7554bcc75-pllvl\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:48.969959 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.969887    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-tmp-dir\") pod \"rate-test-model-8fdd5b-kserve-7554bcc75-pllvl\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:48.969959 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.969954    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-home\") pod \"rate-test-model-8fdd5b-kserve-7554bcc75-pllvl\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:48.970038 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.970015    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-model-cache\") pod \"rate-test-model-8fdd5b-kserve-7554bcc75-pllvl\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:48.971751 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.971729    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-dshm\") pod \"rate-test-model-8fdd5b-kserve-7554bcc75-pllvl\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:48.972125 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.972098    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/897e64e3-fb7c-4a02-9c6f-1012b2810a53-tls-certs\") pod \"rate-test-model-8fdd5b-kserve-7554bcc75-pllvl\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:48.976564 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:48.976537    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfcpx\" (UniqueName: \"kubernetes.io/projected/897e64e3-fb7c-4a02-9c6f-1012b2810a53-kube-api-access-pfcpx\") pod \"rate-test-model-8fdd5b-kserve-7554bcc75-pllvl\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") " pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:49.009234 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:49.009190    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:49.279476 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:49.279449    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"]
Sep 18 06:57:49.280597 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:57:49.280570    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod897e64e3_fb7c_4a02_9c6f_1012b2810a53.slice/crio-9bffb10a6432f342319065c31149ad5d93885764a71a8d808e40ba0d715b2b83 WatchSource:0}: Error finding container 9bffb10a6432f342319065c31149ad5d93885764a71a8d808e40ba0d715b2b83: Status 404 returned error can't find the container with id 9bffb10a6432f342319065c31149ad5d93885764a71a8d808e40ba0d715b2b83
Sep 18 06:57:50.270179 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:50.270135    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl" event={"ID":"897e64e3-fb7c-4a02-9c6f-1012b2810a53","Type":"ContainerStarted","Data":"3cc5782cbbfc1f4ed51b1e7cb6557571f5cec989fcb18c4ea7d50feae3ee5b6e"}
Sep 18 06:57:50.270639 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:50.270188    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl" event={"ID":"897e64e3-fb7c-4a02-9c6f-1012b2810a53","Type":"ContainerStarted","Data":"9bffb10a6432f342319065c31149ad5d93885764a71a8d808e40ba0d715b2b83"}
Sep 18 06:57:50.286366 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:50.286321    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl" podStartSLOduration=2.286306449 podStartE2EDuration="2.286306449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:57:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:57:50.284913903 +0000 UTC m=+2135.966441648" watchObservedRunningTime="2026-09-18 06:57:50.286306449 +0000 UTC m=+2135.967834194"
Sep 18 06:57:52.485327 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:52.485288    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:57:59.010039 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:59.009948    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:59.010039 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:59.010007    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:59.022530 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:59.022505    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:57:59.324311 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:57:59.324232    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:58:24.182316 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:24.182279    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:58:31.034881 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:31.034845    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:58:33.691248 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:33.691200    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:58:39.766870 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:39.766833    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"]
Sep 18 06:58:39.788789 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:39.788759    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:58:40.675538 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:40.675489    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"]
Sep 18 06:58:40.693143 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:40.693116    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:58:41.097229 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:41.097134    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:58:41.384788 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:41.384713    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:58:41.907404 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:41.907376    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:58:44.180852 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:44.180815    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:58:44.428743 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:44.428710    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:58:44.610405 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:44.610327    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:58:44.706255 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:44.706205    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:58:45.233005 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:45.232970    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:58:45.950471 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:45.950443    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:58:46.050423 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:46.050392    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:58:46.150408 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:46.150377    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:58:46.397989 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:46.397959    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:58:46.517056 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:46.517029    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:58:46.656279 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:46.656201    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:58:54.786605 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:54.786543    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd" podUID="11c5e885-4a38-4a83-90d7-6ceeeafff9d1" containerName="main" containerID="cri-o://a1c317f30a04a626836159624f2f493e9b2c7370636621951df47e3f4f38835a" gracePeriod=45
Sep 18 06:58:55.041091 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.041033    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:58:55.103061 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.103023    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-home\") pod \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") "
Sep 18 06:58:55.103292 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.103088    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-tls-certs\") pod \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") "
Sep 18 06:58:55.103292 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.103125    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-dshm\") pod \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") "
Sep 18 06:58:55.103292 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.103242    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-model-cache\") pod \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") "
Sep 18 06:58:55.103506 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.103306    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-tmp-dir\") pod \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") "
Sep 18 06:58:55.103506 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.103354    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sh6cd\" (UniqueName: \"kubernetes.io/projected/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-kube-api-access-sh6cd\") pod \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\" (UID: \"11c5e885-4a38-4a83-90d7-6ceeeafff9d1\") "
Sep 18 06:58:55.103506 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.103346    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-home" (OuterVolumeSpecName: "home") pod "11c5e885-4a38-4a83-90d7-6ceeeafff9d1" (UID: "11c5e885-4a38-4a83-90d7-6ceeeafff9d1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:58:55.103661 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.103503    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-model-cache" (OuterVolumeSpecName: "model-cache") pod "11c5e885-4a38-4a83-90d7-6ceeeafff9d1" (UID: "11c5e885-4a38-4a83-90d7-6ceeeafff9d1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:58:55.103661 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.103558    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "11c5e885-4a38-4a83-90d7-6ceeeafff9d1" (UID: "11c5e885-4a38-4a83-90d7-6ceeeafff9d1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:58:55.103766 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.103723    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:58:55.103825 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.103769    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:58:55.103825 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.103783    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:58:55.105276 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.105251    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "11c5e885-4a38-4a83-90d7-6ceeeafff9d1" (UID: "11c5e885-4a38-4a83-90d7-6ceeeafff9d1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:58:55.105689 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.105667    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-kube-api-access-sh6cd" (OuterVolumeSpecName: "kube-api-access-sh6cd") pod "11c5e885-4a38-4a83-90d7-6ceeeafff9d1" (UID: "11c5e885-4a38-4a83-90d7-6ceeeafff9d1"). InnerVolumeSpecName "kube-api-access-sh6cd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:58:55.105739 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.105715    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-dshm" (OuterVolumeSpecName: "dshm") pod "11c5e885-4a38-4a83-90d7-6ceeeafff9d1" (UID: "11c5e885-4a38-4a83-90d7-6ceeeafff9d1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:58:55.205094 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.205059    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sh6cd\" (UniqueName: \"kubernetes.io/projected/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-kube-api-access-sh6cd\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:58:55.205094 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.205089    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:58:55.205094 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.205099    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/11c5e885-4a38-4a83-90d7-6ceeeafff9d1-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:58:55.622496 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.622463    2568 generic.go:356] "Generic (PLEG): container finished" podID="11c5e885-4a38-4a83-90d7-6ceeeafff9d1" containerID="a1c317f30a04a626836159624f2f493e9b2c7370636621951df47e3f4f38835a" exitCode=0
Sep 18 06:58:55.622679 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.622513    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"
Sep 18 06:58:55.622679 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.622557    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd" event={"ID":"11c5e885-4a38-4a83-90d7-6ceeeafff9d1","Type":"ContainerDied","Data":"a1c317f30a04a626836159624f2f493e9b2c7370636621951df47e3f4f38835a"}
Sep 18 06:58:55.622679 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.622600    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd" event={"ID":"11c5e885-4a38-4a83-90d7-6ceeeafff9d1","Type":"ContainerDied","Data":"55bdddbb82a631ecf80f11e6ef2d5d722089b70705e20cb21d8016cf085368b5"}
Sep 18 06:58:55.622679 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.622618    2568 scope.go:117] "RemoveContainer" containerID="a1c317f30a04a626836159624f2f493e9b2c7370636621951df47e3f4f38835a"
Sep 18 06:58:55.632004 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.631988    2568 scope.go:117] "RemoveContainer" containerID="a1c317f30a04a626836159624f2f493e9b2c7370636621951df47e3f4f38835a"
Sep 18 06:58:55.632321 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:58:55.632300    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1c317f30a04a626836159624f2f493e9b2c7370636621951df47e3f4f38835a\": container with ID starting with a1c317f30a04a626836159624f2f493e9b2c7370636621951df47e3f4f38835a not found: ID does not exist" containerID="a1c317f30a04a626836159624f2f493e9b2c7370636621951df47e3f4f38835a"
Sep 18 06:58:55.632396 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.632330    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1c317f30a04a626836159624f2f493e9b2c7370636621951df47e3f4f38835a"} err="failed to get container status \"a1c317f30a04a626836159624f2f493e9b2c7370636621951df47e3f4f38835a\": rpc error: code = NotFound desc = could not find container \"a1c317f30a04a626836159624f2f493e9b2c7370636621951df47e3f4f38835a\": container with ID starting with a1c317f30a04a626836159624f2f493e9b2c7370636621951df47e3f4f38835a not found: ID does not exist"
Sep 18 06:58:55.644116 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.644092    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"]
Sep 18 06:58:55.647580 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.647559    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ec2077/rate-test-model-ec2077-kserve-84647cd97f-s9xnd"]
Sep 18 06:58:55.694632 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.694608    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl" podUID="897e64e3-fb7c-4a02-9c6f-1012b2810a53" containerName="main" containerID="cri-o://3cc5782cbbfc1f4ed51b1e7cb6557571f5cec989fcb18c4ea7d50feae3ee5b6e" gracePeriod=45
Sep 18 06:58:55.946673 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:55.946650    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:58:56.013551 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.013523    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-tmp-dir\") pod \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") "
Sep 18 06:58:56.013718 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.013575    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-model-cache\") pod \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") "
Sep 18 06:58:56.013718 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.013605    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfcpx\" (UniqueName: \"kubernetes.io/projected/897e64e3-fb7c-4a02-9c6f-1012b2810a53-kube-api-access-pfcpx\") pod \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") "
Sep 18 06:58:56.013718 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.013633    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/897e64e3-fb7c-4a02-9c6f-1012b2810a53-tls-certs\") pod \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") "
Sep 18 06:58:56.013718 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.013667    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-home\") pod \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") "
Sep 18 06:58:56.013950 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.013734    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-dshm\") pod \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\" (UID: \"897e64e3-fb7c-4a02-9c6f-1012b2810a53\") "
Sep 18 06:58:56.013950 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.013812    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "897e64e3-fb7c-4a02-9c6f-1012b2810a53" (UID: "897e64e3-fb7c-4a02-9c6f-1012b2810a53"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:58:56.013950 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.013826    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-model-cache" (OuterVolumeSpecName: "model-cache") pod "897e64e3-fb7c-4a02-9c6f-1012b2810a53" (UID: "897e64e3-fb7c-4a02-9c6f-1012b2810a53"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:58:56.014098 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.013954    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-home" (OuterVolumeSpecName: "home") pod "897e64e3-fb7c-4a02-9c6f-1012b2810a53" (UID: "897e64e3-fb7c-4a02-9c6f-1012b2810a53"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:58:56.014153 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.014130    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:58:56.014153 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.014149    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:58:56.014277 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.014163    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:58:56.015786 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.015769    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-dshm" (OuterVolumeSpecName: "dshm") pod "897e64e3-fb7c-4a02-9c6f-1012b2810a53" (UID: "897e64e3-fb7c-4a02-9c6f-1012b2810a53"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 06:58:56.015786 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.015777    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/897e64e3-fb7c-4a02-9c6f-1012b2810a53-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "897e64e3-fb7c-4a02-9c6f-1012b2810a53" (UID: "897e64e3-fb7c-4a02-9c6f-1012b2810a53"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 06:58:56.015940 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.015924    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/897e64e3-fb7c-4a02-9c6f-1012b2810a53-kube-api-access-pfcpx" (OuterVolumeSpecName: "kube-api-access-pfcpx") pod "897e64e3-fb7c-4a02-9c6f-1012b2810a53" (UID: "897e64e3-fb7c-4a02-9c6f-1012b2810a53"). InnerVolumeSpecName "kube-api-access-pfcpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 06:58:56.115672 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.115640    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfcpx\" (UniqueName: \"kubernetes.io/projected/897e64e3-fb7c-4a02-9c6f-1012b2810a53-kube-api-access-pfcpx\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:58:56.115672 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.115667    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/897e64e3-fb7c-4a02-9c6f-1012b2810a53-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:58:56.115672 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.115677    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/897e64e3-fb7c-4a02-9c6f-1012b2810a53-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 06:58:56.285591 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.285512    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:58:56.543918 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.543839    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11c5e885-4a38-4a83-90d7-6ceeeafff9d1" path="/var/lib/kubelet/pods/11c5e885-4a38-4a83-90d7-6ceeeafff9d1/volumes"
Sep 18 06:58:56.628919 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.628890    2568 generic.go:356] "Generic (PLEG): container finished" podID="897e64e3-fb7c-4a02-9c6f-1012b2810a53" containerID="3cc5782cbbfc1f4ed51b1e7cb6557571f5cec989fcb18c4ea7d50feae3ee5b6e" exitCode=0
Sep 18 06:58:56.629063 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.628926    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"
Sep 18 06:58:56.629063 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.628948    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl" event={"ID":"897e64e3-fb7c-4a02-9c6f-1012b2810a53","Type":"ContainerDied","Data":"3cc5782cbbfc1f4ed51b1e7cb6557571f5cec989fcb18c4ea7d50feae3ee5b6e"}
Sep 18 06:58:56.629063 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.628981    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl" event={"ID":"897e64e3-fb7c-4a02-9c6f-1012b2810a53","Type":"ContainerDied","Data":"9bffb10a6432f342319065c31149ad5d93885764a71a8d808e40ba0d715b2b83"}
Sep 18 06:58:56.629063 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.629001    2568 scope.go:117] "RemoveContainer" containerID="3cc5782cbbfc1f4ed51b1e7cb6557571f5cec989fcb18c4ea7d50feae3ee5b6e"
Sep 18 06:58:56.637618 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.637599    2568 scope.go:117] "RemoveContainer" containerID="3cc5782cbbfc1f4ed51b1e7cb6557571f5cec989fcb18c4ea7d50feae3ee5b6e"
Sep 18 06:58:56.637897 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:58:56.637878    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cc5782cbbfc1f4ed51b1e7cb6557571f5cec989fcb18c4ea7d50feae3ee5b6e\": container with ID starting with 3cc5782cbbfc1f4ed51b1e7cb6557571f5cec989fcb18c4ea7d50feae3ee5b6e not found: ID does not exist" containerID="3cc5782cbbfc1f4ed51b1e7cb6557571f5cec989fcb18c4ea7d50feae3ee5b6e"
Sep 18 06:58:56.637950 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.637910    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cc5782cbbfc1f4ed51b1e7cb6557571f5cec989fcb18c4ea7d50feae3ee5b6e"} err="failed to get container status \"3cc5782cbbfc1f4ed51b1e7cb6557571f5cec989fcb18c4ea7d50feae3ee5b6e\": rpc error: code = NotFound desc = could not find container \"3cc5782cbbfc1f4ed51b1e7cb6557571f5cec989fcb18c4ea7d50feae3ee5b6e\": container with ID starting with 3cc5782cbbfc1f4ed51b1e7cb6557571f5cec989fcb18c4ea7d50feae3ee5b6e not found: ID does not exist"
Sep 18 06:58:56.645055 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.645030    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"]
Sep 18 06:58:56.646880 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:56.646863    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-8fdd5b/rate-test-model-8fdd5b-kserve-7554bcc75-pllvl"]
Sep 18 06:58:58.548630 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:58:58.548599    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="897e64e3-fb7c-4a02-9c6f-1012b2810a53" path="/var/lib/kubelet/pods/897e64e3-fb7c-4a02-9c6f-1012b2810a53/volumes"
Sep 18 06:59:06.574374 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:06.574333    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:08.062311 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:08.062279    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:09.334418 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:09.334381    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:09.537833 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:09.537804    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:10.049280 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.049245    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:10.258194 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.258147    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:10.512844 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.512804    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"]
Sep 18 06:59:10.513454 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.513426    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11c5e885-4a38-4a83-90d7-6ceeeafff9d1" containerName="main"
Sep 18 06:59:10.513454 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.513453    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="11c5e885-4a38-4a83-90d7-6ceeeafff9d1" containerName="main"
Sep 18 06:59:10.513671 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.513479    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="897e64e3-fb7c-4a02-9c6f-1012b2810a53" containerName="main"
Sep 18 06:59:10.513671 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.513488    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="897e64e3-fb7c-4a02-9c6f-1012b2810a53" containerName="main"
Sep 18 06:59:10.513671 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.513598    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="897e64e3-fb7c-4a02-9c6f-1012b2810a53" containerName="main"
Sep 18 06:59:10.513671 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.513607    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="11c5e885-4a38-4a83-90d7-6ceeeafff9d1" containerName="main"
Sep 18 06:59:10.516510 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.516493    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.518764 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.518745    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-1672e6-openshift-default-dockercfg-dpnk6\""
Sep 18 06:59:10.527097 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.527075    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"]
Sep 18 06:59:10.543048 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.543016    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-workload-socket\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.543206 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.543081    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsvzh\" (UniqueName: \"kubernetes.io/projected/21b5217f-fb47-4ddd-9416-1a508288f4ac-kube-api-access-zsvzh\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.543206 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.543139    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-credential-socket\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.543206 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.543165    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-podinfo\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.543206 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.543195    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-workload-certs\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.543453 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.543262    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-data\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.543453 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.543288    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21b5217f-fb47-4ddd-9416-1a508288f4ac-istiod-ca-cert\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.543453 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.543316    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-token\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.543613 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.543474    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-envoy\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.546101 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.546080    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:10.644635 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.644596    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-envoy\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.644814 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.644669    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-workload-socket\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.644814 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.644693    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsvzh\" (UniqueName: \"kubernetes.io/projected/21b5217f-fb47-4ddd-9416-1a508288f4ac-kube-api-access-zsvzh\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.644814 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.644732    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-credential-socket\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.644814 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.644760    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-podinfo\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.644814 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.644787    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-workload-certs\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.645098 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.644821    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-data\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.645098 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.644849    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21b5217f-fb47-4ddd-9416-1a508288f4ac-istiod-ca-cert\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.645098 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.644878    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-token\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.645279 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.645128    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-credential-socket\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.645382 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.645357    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-data\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.645558 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.645427    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-workload-certs\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.645678 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.645613    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-workload-socket\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.645901 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.645878    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21b5217f-fb47-4ddd-9416-1a508288f4ac-istiod-ca-cert\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.647469 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.647451    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-envoy\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.647738 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.647710    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-podinfo\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.656009 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.655981    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-token\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.656146 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.656130    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsvzh\" (UniqueName: \"kubernetes.io/projected/21b5217f-fb47-4ddd-9416-1a508288f4ac-kube-api-access-zsvzh\") pod \"e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") " pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.828465 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.828386    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:10.928943 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:10.928909    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:11.023902 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:11.023869    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:11.103197 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:11.103172    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"]
Sep 18 06:59:11.105321 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:59:11.105294    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21b5217f_fb47_4ddd_9416_1a508288f4ac.slice/crio-3495adca49b11dd1327871c5b034a512f966133dad0613e2d75f358a9968718e WatchSource:0}: Error finding container 3495adca49b11dd1327871c5b034a512f966133dad0613e2d75f358a9968718e: Status 404 returned error can't find the container with id 3495adca49b11dd1327871c5b034a512f966133dad0613e2d75f358a9968718e
Sep 18 06:59:11.107546 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:11.107513    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:59:11.107665 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:11.107590    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:59:11.107665 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:11.107633    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:59:11.594905 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:11.594875    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:11.705031 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:11.705001    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts" event={"ID":"21b5217f-fb47-4ddd-9416-1a508288f4ac","Type":"ContainerStarted","Data":"82555b00fe926571204d222c65c829ba65a2f409cb9d5549bc965e0ba9801b3f"}
Sep 18 06:59:11.705203 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:11.705054    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts" event={"ID":"21b5217f-fb47-4ddd-9416-1a508288f4ac","Type":"ContainerStarted","Data":"3495adca49b11dd1327871c5b034a512f966133dad0613e2d75f358a9968718e"}
Sep 18 06:59:11.725590 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:11.725545    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts" podStartSLOduration=1.7255304329999999 podStartE2EDuration="1.725530433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:59:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:59:11.722961621 +0000 UTC m=+2217.404489366" watchObservedRunningTime="2026-09-18 06:59:11.725530433 +0000 UTC m=+2217.407058177"
Sep 18 06:59:11.828583 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:11.828537    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:12.836342 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:12.836310    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:12.934701 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:12.934670    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:13.647609 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:13.647580    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:13.715100 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:13.715070    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:13.715885 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:13.715867    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:13.743009 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:13.742979    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:14.340491 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:14.340429    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:15.157776 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:15.157733    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:15.807771 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:15.807201    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"]
Sep 18 06:59:15.810978 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:15.810956    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"
Sep 18 06:59:15.813555 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:15.813532    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 06:59:15.813673 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:15.813571    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 18 06:59:15.814821 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:15.814802    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-1672e6-serving-cert\""
Sep 18 06:59:15.815126 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:15.815109    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 18 06:59:15.815446 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:15.815423    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-mxmzc\""
Sep 18 06:59:15.825422 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:15.825006    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"]
Sep 18 06:59:15.846550 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:15.846518    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:15.895447 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:15.895416    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1537d085-ca41-4bbd-9fa2-9b415ec259a4-metrics-certs\") pod \"maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt\" (UID: \"1537d085-ca41-4bbd-9fa2-9b415ec259a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"
Sep 18 06:59:15.895595 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:15.895492    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7b6m\" (UniqueName: \"kubernetes.io/projected/1537d085-ca41-4bbd-9fa2-9b415ec259a4-kube-api-access-n7b6m\") pod \"maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt\" (UID: \"1537d085-ca41-4bbd-9fa2-9b415ec259a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"
Sep 18 06:59:15.895595 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:15.895549    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1537d085-ca41-4bbd-9fa2-9b415ec259a4-maas-api-tls\") pod \"maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt\" (UID: \"1537d085-ca41-4bbd-9fa2-9b415ec259a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"
Sep 18 06:59:15.895595 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:15.895572    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1537d085-ca41-4bbd-9fa2-9b415ec259a4-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt\" (UID: \"1537d085-ca41-4bbd-9fa2-9b415ec259a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"
Sep 18 06:59:15.950361 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:15.950327    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:15.996073 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:15.996043    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1537d085-ca41-4bbd-9fa2-9b415ec259a4-metrics-certs\") pod \"maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt\" (UID: \"1537d085-ca41-4bbd-9fa2-9b415ec259a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"
Sep 18 06:59:15.996276 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:15.996109    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7b6m\" (UniqueName: \"kubernetes.io/projected/1537d085-ca41-4bbd-9fa2-9b415ec259a4-kube-api-access-n7b6m\") pod \"maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt\" (UID: \"1537d085-ca41-4bbd-9fa2-9b415ec259a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"
Sep 18 06:59:15.996276 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:15.996149    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1537d085-ca41-4bbd-9fa2-9b415ec259a4-maas-api-tls\") pod \"maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt\" (UID: \"1537d085-ca41-4bbd-9fa2-9b415ec259a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"
Sep 18 06:59:15.996276 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:15.996182    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1537d085-ca41-4bbd-9fa2-9b415ec259a4-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt\" (UID: \"1537d085-ca41-4bbd-9fa2-9b415ec259a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"
Sep 18 06:59:15.996433 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:59:15.996281    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-serving-cert: secret "maas-api-e2e-ipp-b-1672e6-serving-cert" not found
Sep 18 06:59:15.996433 ip-10-0-128-30 kubenswrapper[2568]: E0918 06:59:15.996349    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1537d085-ca41-4bbd-9fa2-9b415ec259a4-maas-api-tls podName:1537d085-ca41-4bbd-9fa2-9b415ec259a4 nodeName:}" failed. No retries permitted until 2026-09-18 06:59:16.496329236 +0000 UTC m=+2222.177856973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1537d085-ca41-4bbd-9fa2-9b415ec259a4-maas-api-tls") pod "maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt" (UID: "1537d085-ca41-4bbd-9fa2-9b415ec259a4") : secret "maas-api-e2e-ipp-b-1672e6-serving-cert" not found
Sep 18 06:59:15.996895 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:15.996878    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1537d085-ca41-4bbd-9fa2-9b415ec259a4-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt\" (UID: \"1537d085-ca41-4bbd-9fa2-9b415ec259a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"
Sep 18 06:59:15.998622 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:15.998598    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1537d085-ca41-4bbd-9fa2-9b415ec259a4-metrics-certs\") pod \"maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt\" (UID: \"1537d085-ca41-4bbd-9fa2-9b415ec259a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"
Sep 18 06:59:16.004620 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:16.004600    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7b6m\" (UniqueName: \"kubernetes.io/projected/1537d085-ca41-4bbd-9fa2-9b415ec259a4-kube-api-access-n7b6m\") pod \"maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt\" (UID: \"1537d085-ca41-4bbd-9fa2-9b415ec259a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"
Sep 18 06:59:16.147731 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:16.147700    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:16.376797 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:16.376766    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:16.458984 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:16.458898    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:16.502512 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:16.502478    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1537d085-ca41-4bbd-9fa2-9b415ec259a4-maas-api-tls\") pod \"maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt\" (UID: \"1537d085-ca41-4bbd-9fa2-9b415ec259a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"
Sep 18 06:59:16.505030 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:16.505009    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1537d085-ca41-4bbd-9fa2-9b415ec259a4-maas-api-tls\") pod \"maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt\" (UID: \"1537d085-ca41-4bbd-9fa2-9b415ec259a4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"
Sep 18 06:59:16.728554 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:16.728444    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-mxmzc\""
Sep 18 06:59:16.736342 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:16.736314    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"
Sep 18 06:59:17.005699 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:17.005673    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"]
Sep 18 06:59:17.007682 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:59:17.007657    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1537d085_ca41_4bbd_9fa2_9b415ec259a4.slice/crio-466edb7de792d80a6c4ea0c102634d84bebd99704f779c590f2e63ea11a97dcc WatchSource:0}: Error finding container 466edb7de792d80a6c4ea0c102634d84bebd99704f779c590f2e63ea11a97dcc: Status 404 returned error can't find the container with id 466edb7de792d80a6c4ea0c102634d84bebd99704f779c590f2e63ea11a97dcc
Sep 18 06:59:17.015991 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:17.015969    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 06:59:17.739077 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:17.739039    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt" event={"ID":"1537d085-ca41-4bbd-9fa2-9b415ec259a4","Type":"ContainerStarted","Data":"300b6320962acfb1801042844dd960cccf607d1a4d9d5f5cecfbd9bf224adef4"}
Sep 18 06:59:17.739305 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:17.739133    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"
Sep 18 06:59:17.739305 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:17.739153    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt" event={"ID":"1537d085-ca41-4bbd-9fa2-9b415ec259a4","Type":"ContainerStarted","Data":"466edb7de792d80a6c4ea0c102634d84bebd99704f779c590f2e63ea11a97dcc"}
Sep 18 06:59:17.756533 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:17.756490    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt" podStartSLOduration=2.425586312 podStartE2EDuration="2.75647676s" totalImagesPullingTime="330.890448ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:59:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:59:17.754969241 +0000 UTC m=+2223.436496982" watchObservedRunningTime="2026-09-18 06:59:17.75647676 +0000 UTC m=+2223.438004504"
Sep 18 06:59:22.094142 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:22.094110    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 06:59:23.751002 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:23.750970    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"
Sep 18 06:59:32.814847 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:32.814821    2568 generic.go:356] "Generic (PLEG): container finished" podID="21b5217f-fb47-4ddd-9416-1a508288f4ac" containerID="82555b00fe926571204d222c65c829ba65a2f409cb9d5549bc965e0ba9801b3f" exitCode=0
Sep 18 06:59:32.815207 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:32.814907    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts" event={"ID":"21b5217f-fb47-4ddd-9416-1a508288f4ac","Type":"ContainerDied","Data":"82555b00fe926571204d222c65c829ba65a2f409cb9d5549bc965e0ba9801b3f"}
Sep 18 06:59:32.815381 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:32.815366    2568 scope.go:117] "RemoveContainer" containerID="82555b00fe926571204d222c65c829ba65a2f409cb9d5549bc965e0ba9801b3f"
Sep 18 06:59:32.816451 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:32.816426    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:59:32.816512 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:32.816493    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:59:32.816558 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:32.816526    2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 18 06:59:33.821780 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:33.821740    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts" event={"ID":"21b5217f-fb47-4ddd-9416-1a508288f4ac","Type":"ContainerStarted","Data":"c807873f38efea4c856642cb70ba2b509e799a364c20317e43561bb669c34771"}
Sep 18 06:59:33.828641 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:33.828624    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:33.828817 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:33.828798    2568 patch_prober.go:28] interesting pod/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Sep 18 06:59:33.828870 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:33.828853    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts" podUID="21b5217f-fb47-4ddd-9416-1a508288f4ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 18 06:59:34.829130 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:34.829095    2568 patch_prober.go:28] interesting pod/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Sep 18 06:59:34.829669 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:34.829147    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts" podUID="21b5217f-fb47-4ddd-9416-1a508288f4ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 18 06:59:35.828800 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:35.828759    2568 patch_prober.go:28] interesting pod/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Sep 18 06:59:35.829014 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:35.828834    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts" podUID="21b5217f-fb47-4ddd-9416-1a508288f4ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 18 06:59:36.829392 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:36.829350    2568 patch_prober.go:28] interesting pod/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Sep 18 06:59:36.829926 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:36.829419    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts" podUID="21b5217f-fb47-4ddd-9416-1a508288f4ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 18 06:59:37.828885 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:37.828848    2568 patch_prober.go:28] interesting pod/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Sep 18 06:59:37.829074 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:37.828918    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts" podUID="21b5217f-fb47-4ddd-9416-1a508288f4ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 18 06:59:38.701464 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:38.701431    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:38.829028 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:38.828996    2568 patch_prober.go:28] interesting pod/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body=
Sep 18 06:59:38.829209 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:38.829062    2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts" podUID="21b5217f-fb47-4ddd-9416-1a508288f4ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Sep 18 06:59:39.427504 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:39.427461    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:39.833491 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:39.833412    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:39.833840 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:39.833760    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:39.834526 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:39.834510    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 06:59:40.060117 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:40.060081    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:40.678054 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:40.678024    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:41.438515 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:41.438483    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:42.039074 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:42.039037    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:43.699898 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:43.699849    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:55.888861 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:55.888825    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"]
Sep 18 06:59:55.892053 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:55.892030    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 06:59:55.895562 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:55.895540    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-a7bd58\"/\"kube-root-ca.crt\""
Sep 18 06:59:55.895710 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:55.895541    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-a7bd58\"/\"default-dockercfg-6xd7p\""
Sep 18 06:59:55.895710 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:55.895542    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-a7bd58\"/\"ipp-route-a7bd58-kserve-self-signed-certs\""
Sep 18 06:59:55.895710 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:55.895569    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-a7bd58\"/\"openshift-service-ca.crt\""
Sep 18 06:59:55.902103 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:55.902080    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"]
Sep 18 06:59:55.910434 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:55.910413    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 06:59:56.003035 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.003006    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-dshm\") pod \"ipp-route-a7bd58-kserve-5966755cc5-hz9tn\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") " pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 06:59:56.003188 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.003043    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjc6n\" (UniqueName: \"kubernetes.io/projected/8f384570-0092-479f-8afc-a50be435b3bd-kube-api-access-jjc6n\") pod \"ipp-route-a7bd58-kserve-5966755cc5-hz9tn\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") " pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 06:59:56.003188 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.003162    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f384570-0092-479f-8afc-a50be435b3bd-tls-certs\") pod \"ipp-route-a7bd58-kserve-5966755cc5-hz9tn\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") " pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 06:59:56.003302 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.003192    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-home\") pod \"ipp-route-a7bd58-kserve-5966755cc5-hz9tn\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") " pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 06:59:56.003302 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.003244    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-tmp-dir\") pod \"ipp-route-a7bd58-kserve-5966755cc5-hz9tn\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") " pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 06:59:56.003302 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.003277    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-model-cache\") pod \"ipp-route-a7bd58-kserve-5966755cc5-hz9tn\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") " pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 06:59:56.104403 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.104364    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjc6n\" (UniqueName: \"kubernetes.io/projected/8f384570-0092-479f-8afc-a50be435b3bd-kube-api-access-jjc6n\") pod \"ipp-route-a7bd58-kserve-5966755cc5-hz9tn\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") " pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 06:59:56.104609 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.104463    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f384570-0092-479f-8afc-a50be435b3bd-tls-certs\") pod \"ipp-route-a7bd58-kserve-5966755cc5-hz9tn\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") " pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 06:59:56.104609 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.104483    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-home\") pod \"ipp-route-a7bd58-kserve-5966755cc5-hz9tn\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") " pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 06:59:56.104609 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.104513    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-tmp-dir\") pod \"ipp-route-a7bd58-kserve-5966755cc5-hz9tn\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") " pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 06:59:56.104609 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.104536    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-model-cache\") pod \"ipp-route-a7bd58-kserve-5966755cc5-hz9tn\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") " pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 06:59:56.104609 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.104585    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-dshm\") pod \"ipp-route-a7bd58-kserve-5966755cc5-hz9tn\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") " pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 06:59:56.104978 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.104950    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-home\") pod \"ipp-route-a7bd58-kserve-5966755cc5-hz9tn\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") " pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 06:59:56.104978 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.104968    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-tmp-dir\") pod \"ipp-route-a7bd58-kserve-5966755cc5-hz9tn\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") " pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 06:59:56.105075 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.105021    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-model-cache\") pod \"ipp-route-a7bd58-kserve-5966755cc5-hz9tn\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") " pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 06:59:56.106845 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.106824    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-dshm\") pod \"ipp-route-a7bd58-kserve-5966755cc5-hz9tn\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") " pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 06:59:56.107288 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.107270    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f384570-0092-479f-8afc-a50be435b3bd-tls-certs\") pod \"ipp-route-a7bd58-kserve-5966755cc5-hz9tn\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") " pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 06:59:56.111566 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.111542    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjc6n\" (UniqueName: \"kubernetes.io/projected/8f384570-0092-479f-8afc-a50be435b3bd-kube-api-access-jjc6n\") pod \"ipp-route-a7bd58-kserve-5966755cc5-hz9tn\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") " pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 06:59:56.202702 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.202671    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 06:59:56.472542 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.472519    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"]
Sep 18 06:59:56.474140 ip-10-0-128-30 kubenswrapper[2568]: W0918 06:59:56.474107    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f384570_0092_479f_8afc_a50be435b3bd.slice/crio-013abe1cdcbc8be65708226cecce0c8a56628f0cfe33d1aeaff1c8ca2c9f0e2c WatchSource:0}: Error finding container 013abe1cdcbc8be65708226cecce0c8a56628f0cfe33d1aeaff1c8ca2c9f0e2c: Status 404 returned error can't find the container with id 013abe1cdcbc8be65708226cecce0c8a56628f0cfe33d1aeaff1c8ca2c9f0e2c
Sep 18 06:59:56.952428 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.952392    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn" event={"ID":"8f384570-0092-479f-8afc-a50be435b3bd","Type":"ContainerStarted","Data":"e9ea90cdf0bbd08f134bc3b49758fb12c493442dba94f9b435a3b33fb7d0289a"}
Sep 18 06:59:56.952910 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.952443    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn" event={"ID":"8f384570-0092-479f-8afc-a50be435b3bd","Type":"ContainerStarted","Data":"013abe1cdcbc8be65708226cecce0c8a56628f0cfe33d1aeaff1c8ca2c9f0e2c"}
Sep 18 06:59:56.968917 ip-10-0-128-30 kubenswrapper[2568]: I0918 06:59:56.968873    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn" podStartSLOduration=1.9688585779999999 podStartE2EDuration="1.968858578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 06:59:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 06:59:56.96683691 +0000 UTC m=+2262.648364657" watchObservedRunningTime="2026-09-18 06:59:56.968858578 +0000 UTC m=+2262.650386323"
Sep 18 07:00:00.135656 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:00.135612    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ec2077-29828580-c9sgv"]
Sep 18 07:00:00.138577 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:00.138555    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ec2077-29828580-c9sgv"
Sep 18 07:00:00.138822 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:00.138797    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-8fdd5b-29828580-mmprx"]
Sep 18 07:00:00.141021 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:00.140997    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-88t7h\""
Sep 18 07:00:00.142133 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:00.142109    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-8fdd5b-29828580-mmprx"
Sep 18 07:00:00.150557 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:00.150532    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ec2077-29828580-c9sgv"]
Sep 18 07:00:00.153441 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:00.153416    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-8fdd5b-29828580-mmprx"]
Sep 18 07:00:00.156393 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:00.156370    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ec2077-29828580-c9sgv"
Sep 18 07:00:00.164504 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:00.164483    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-8fdd5b-29828580-mmprx"
Sep 18 07:00:00.433288 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:00.433257    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ec2077-29828580-c9sgv"]
Sep 18 07:00:00.434799 ip-10-0-128-30 kubenswrapper[2568]: W0918 07:00:00.434764    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa3965d7_7b89_4bd5_9091_8ec8a6183ef9.slice/crio-49c8922bf6fd4f7c4a618c7ef06d15efd1823cecd740194e57ec0dcd98ce16fe WatchSource:0}: Error finding container 49c8922bf6fd4f7c4a618c7ef06d15efd1823cecd740194e57ec0dcd98ce16fe: Status 404 returned error can't find the container with id 49c8922bf6fd4f7c4a618c7ef06d15efd1823cecd740194e57ec0dcd98ce16fe
Sep 18 07:00:00.455690 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:00.455665    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-8fdd5b-29828580-mmprx"]
Sep 18 07:00:00.457063 ip-10-0-128-30 kubenswrapper[2568]: W0918 07:00:00.457032    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32e453be_f3ab_4ef9_9cb7_b886ec15a3c6.slice/crio-f86332965e9bc18e1713502537412faad29d6062af25885e4d1264660a0d5757 WatchSource:0}: Error finding container f86332965e9bc18e1713502537412faad29d6062af25885e4d1264660a0d5757: Status 404 returned error can't find the container with id f86332965e9bc18e1713502537412faad29d6062af25885e4d1264660a0d5757
Sep 18 07:00:00.497500 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:00.497460    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:00:00.976811 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:00.976762    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-8fdd5b-29828580-mmprx" event={"ID":"32e453be-f3ab-4ef9-9cb7-b886ec15a3c6","Type":"ContainerStarted","Data":"164d40bfa4bb0a5d3f956efcf9282f4c2d3231aeb8d00fd3c12ecf76634e9db4"}
Sep 18 07:00:00.977038 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:00.976828    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-8fdd5b-29828580-mmprx" event={"ID":"32e453be-f3ab-4ef9-9cb7-b886ec15a3c6","Type":"ContainerStarted","Data":"f86332965e9bc18e1713502537412faad29d6062af25885e4d1264660a0d5757"}
Sep 18 07:00:00.978297 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:00.978276    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ec2077-29828580-c9sgv" event={"ID":"aa3965d7-7b89-4bd5-9091-8ec8a6183ef9","Type":"ContainerStarted","Data":"1625768dcbbcc678e52c61f1de83047f529ea95caccd505edf77aa9ade6f5d9c"}
Sep 18 07:00:00.978370 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:00.978320    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ec2077-29828580-c9sgv" event={"ID":"aa3965d7-7b89-4bd5-9091-8ec8a6183ef9","Type":"ContainerStarted","Data":"49c8922bf6fd4f7c4a618c7ef06d15efd1823cecd740194e57ec0dcd98ce16fe"}
Sep 18 07:00:00.999582 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:00.999485    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-8fdd5b-29828580-mmprx" podStartSLOduration=0.999466087 podStartE2EDuration="999.466087ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 07:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 07:00:00.997691922 +0000 UTC m=+2266.679219682" watchObservedRunningTime="2026-09-18 07:00:00.999466087 +0000 UTC m=+2266.680993827"
Sep 18 07:00:01.016805 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:01.016747    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ec2077-29828580-c9sgv" podStartSLOduration=1.016728461 podStartE2EDuration="1.016728461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 07:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 07:00:01.016156838 +0000 UTC m=+2266.697684582" watchObservedRunningTime="2026-09-18 07:00:01.016728461 +0000 UTC m=+2266.698256207"
Sep 18 07:00:01.985415 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:01.985377    2568 generic.go:356] "Generic (PLEG): container finished" podID="32e453be-f3ab-4ef9-9cb7-b886ec15a3c6" containerID="164d40bfa4bb0a5d3f956efcf9282f4c2d3231aeb8d00fd3c12ecf76634e9db4" exitCode=0
Sep 18 07:00:01.985884 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:01.985457    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-8fdd5b-29828580-mmprx" event={"ID":"32e453be-f3ab-4ef9-9cb7-b886ec15a3c6","Type":"ContainerDied","Data":"164d40bfa4bb0a5d3f956efcf9282f4c2d3231aeb8d00fd3c12ecf76634e9db4"}
Sep 18 07:00:01.987072 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:01.987047    2568 generic.go:356] "Generic (PLEG): container finished" podID="aa3965d7-7b89-4bd5-9091-8ec8a6183ef9" containerID="1625768dcbbcc678e52c61f1de83047f529ea95caccd505edf77aa9ade6f5d9c" exitCode=0
Sep 18 07:00:01.987210 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:01.987124    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ec2077-29828580-c9sgv" event={"ID":"aa3965d7-7b89-4bd5-9091-8ec8a6183ef9","Type":"ContainerDied","Data":"1625768dcbbcc678e52c61f1de83047f529ea95caccd505edf77aa9ade6f5d9c"}
Sep 18 07:00:03.151925 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:03.151900    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-8fdd5b-29828580-mmprx"
Sep 18 07:00:03.154765 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:03.154742    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ec2077-29828580-c9sgv"
Sep 18 07:00:03.998410 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:03.998380    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-8fdd5b-29828580-mmprx"
Sep 18 07:00:03.998410 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:03.998415    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-8fdd5b-29828580-mmprx" event={"ID":"32e453be-f3ab-4ef9-9cb7-b886ec15a3c6","Type":"ContainerDied","Data":"f86332965e9bc18e1713502537412faad29d6062af25885e4d1264660a0d5757"}
Sep 18 07:00:03.998668 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:03.998460    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f86332965e9bc18e1713502537412faad29d6062af25885e4d1264660a0d5757"
Sep 18 07:00:03.999739 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:03.999717    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ec2077-29828580-c9sgv"
Sep 18 07:00:03.999866 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:03.999768    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ec2077-29828580-c9sgv" event={"ID":"aa3965d7-7b89-4bd5-9091-8ec8a6183ef9","Type":"ContainerDied","Data":"49c8922bf6fd4f7c4a618c7ef06d15efd1823cecd740194e57ec0dcd98ce16fe"}
Sep 18 07:00:03.999866 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:03.999815    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49c8922bf6fd4f7c4a618c7ef06d15efd1823cecd740194e57ec0dcd98ce16fe"
Sep 18 07:00:06.203193 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:06.203156    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 07:00:06.203193 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:06.203202    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 07:00:06.215836 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:06.215809    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 07:00:07.026586 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:07.026559    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 07:00:13.895380 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:13.895344    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:00:20.160826 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:20.160789    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:20.452692 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:20.452587    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:20.741649 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:20.741552    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:22.459003 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:22.458964    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:23.234022 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:23.233988    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:23.324205 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:23.324173    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:24.020331 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:24.020296    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:24.837632 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:24.837573    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:25.534588 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:25.534556    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:26.160638 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:26.160607    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:26.262085 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:26.262051    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:26.363724 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:26.363690    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:26.613122 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:26.613038    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:26.720958 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:26.720921    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:32.486629 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:32.486591    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:00:48.524567 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:48.524525    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:49.125738 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:49.125701    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:49.790006 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:49.789967    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:50.424016 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:50.423984    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:51.108117 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:51.108081    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:51.752970 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:51.752937    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:53.314267 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:53.314208    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:54.983892 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:54.983858    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:55.485680 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:55.485637    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:56.361865 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:56.361834    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:57.013006 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:57.012976    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:57.779314 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:57.779279    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:00:58.326147 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:00:58.326115    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:04.889356 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:04.889322    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:04.994901 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:04.994870    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:05.095918 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:05.095881    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:05.195943 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:05.195911    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:05.484815 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:05.484737    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:05.585211 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:05.585176    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:06.273567 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:06.273530    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:07.118594 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:07.118559    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:07.646470 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:07.646434    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:08.287446 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:08.287415    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:09.380198 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:09.380165    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:01:11.146841 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:11.146803    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:11.210007 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:11.209947    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:11.394373 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:11.394346    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:15.005579 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:15.005547    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:15.800142 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:15.800107    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:16.513417 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:16.513366    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:16.890545 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:16.890514    2568 scope.go:117] "RemoveContainer" containerID="3192af24a8c4c714a1352f04c1a1ad5fb797f6c65b14979176e2cb4be134ffbf"
Sep 18 07:01:17.158721 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:17.158692    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:17.814056 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:17.814021    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:18.537799 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:18.537763    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:27.884793 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:27.884757    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:01:30.343213 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:30.343180    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:30.442424 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:30.442386    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:30.512955 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:30.512926    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:30.565434 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:30.565405    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:30.858537 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:30.858506    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:30.976634 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:30.976601    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:31.634717 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:31.634687    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:32.239887 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:32.239856    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:33.013045 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:33.013008    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:35.858751 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:35.858715    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"]
Sep 18 07:01:35.902140 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:35.902107    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:36.125172 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:36.125091    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:01:41.287797 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:41.287759    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:01:50.878696 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:50.878655    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn" podUID="8f384570-0092-479f-8afc-a50be435b3bd" containerName="main" containerID="cri-o://e9ea90cdf0bbd08f134bc3b49758fb12c493442dba94f9b435a3b33fb7d0289a" gracePeriod=45
Sep 18 07:01:51.125515 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.125488    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 07:01:51.250859 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.250829    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-tmp-dir\") pod \"8f384570-0092-479f-8afc-a50be435b3bd\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") "
Sep 18 07:01:51.251065 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.250885    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-dshm\") pod \"8f384570-0092-479f-8afc-a50be435b3bd\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") "
Sep 18 07:01:51.251065 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.250933    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-model-cache\") pod \"8f384570-0092-479f-8afc-a50be435b3bd\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") "
Sep 18 07:01:51.251065 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.250968    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f384570-0092-479f-8afc-a50be435b3bd-tls-certs\") pod \"8f384570-0092-479f-8afc-a50be435b3bd\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") "
Sep 18 07:01:51.251065 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.251028    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjc6n\" (UniqueName: \"kubernetes.io/projected/8f384570-0092-479f-8afc-a50be435b3bd-kube-api-access-jjc6n\") pod \"8f384570-0092-479f-8afc-a50be435b3bd\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") "
Sep 18 07:01:51.251065 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.251053    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-home\") pod \"8f384570-0092-479f-8afc-a50be435b3bd\" (UID: \"8f384570-0092-479f-8afc-a50be435b3bd\") "
Sep 18 07:01:51.251353 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.251168    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8f384570-0092-479f-8afc-a50be435b3bd" (UID: "8f384570-0092-479f-8afc-a50be435b3bd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:01:51.251353 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.251182    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-model-cache" (OuterVolumeSpecName: "model-cache") pod "8f384570-0092-479f-8afc-a50be435b3bd" (UID: "8f384570-0092-479f-8afc-a50be435b3bd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:01:51.251455 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.251402    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-home" (OuterVolumeSpecName: "home") pod "8f384570-0092-479f-8afc-a50be435b3bd" (UID: "8f384570-0092-479f-8afc-a50be435b3bd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:01:51.251455 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.251432    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:01:51.251455 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.251446    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:01:51.253264 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.253232    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f384570-0092-479f-8afc-a50be435b3bd-kube-api-access-jjc6n" (OuterVolumeSpecName: "kube-api-access-jjc6n") pod "8f384570-0092-479f-8afc-a50be435b3bd" (UID: "8f384570-0092-479f-8afc-a50be435b3bd"). InnerVolumeSpecName "kube-api-access-jjc6n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 07:01:51.253385 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.253283    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-dshm" (OuterVolumeSpecName: "dshm") pod "8f384570-0092-479f-8afc-a50be435b3bd" (UID: "8f384570-0092-479f-8afc-a50be435b3bd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:01:51.253385 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.253345    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f384570-0092-479f-8afc-a50be435b3bd-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8f384570-0092-479f-8afc-a50be435b3bd" (UID: "8f384570-0092-479f-8afc-a50be435b3bd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 07:01:51.351962 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.351929    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjc6n\" (UniqueName: \"kubernetes.io/projected/8f384570-0092-479f-8afc-a50be435b3bd-kube-api-access-jjc6n\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:01:51.351962 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.351956    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:01:51.351962 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.351969    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f384570-0092-479f-8afc-a50be435b3bd-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:01:51.352181 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.351977    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f384570-0092-479f-8afc-a50be435b3bd-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:01:51.586736 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.586652    2568 generic.go:356] "Generic (PLEG): container finished" podID="8f384570-0092-479f-8afc-a50be435b3bd" containerID="e9ea90cdf0bbd08f134bc3b49758fb12c493442dba94f9b435a3b33fb7d0289a" exitCode=0
Sep 18 07:01:51.586736 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.586695    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"
Sep 18 07:01:51.586928 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.586738    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn" event={"ID":"8f384570-0092-479f-8afc-a50be435b3bd","Type":"ContainerDied","Data":"e9ea90cdf0bbd08f134bc3b49758fb12c493442dba94f9b435a3b33fb7d0289a"}
Sep 18 07:01:51.586928 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.586778    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn" event={"ID":"8f384570-0092-479f-8afc-a50be435b3bd","Type":"ContainerDied","Data":"013abe1cdcbc8be65708226cecce0c8a56628f0cfe33d1aeaff1c8ca2c9f0e2c"}
Sep 18 07:01:51.586928 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.586797    2568 scope.go:117] "RemoveContainer" containerID="e9ea90cdf0bbd08f134bc3b49758fb12c493442dba94f9b435a3b33fb7d0289a"
Sep 18 07:01:51.596328 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.596308    2568 scope.go:117] "RemoveContainer" containerID="e9ea90cdf0bbd08f134bc3b49758fb12c493442dba94f9b435a3b33fb7d0289a"
Sep 18 07:01:51.596616 ip-10-0-128-30 kubenswrapper[2568]: E0918 07:01:51.596600    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9ea90cdf0bbd08f134bc3b49758fb12c493442dba94f9b435a3b33fb7d0289a\": container with ID starting with e9ea90cdf0bbd08f134bc3b49758fb12c493442dba94f9b435a3b33fb7d0289a not found: ID does not exist" containerID="e9ea90cdf0bbd08f134bc3b49758fb12c493442dba94f9b435a3b33fb7d0289a"
Sep 18 07:01:51.596668 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.596624    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9ea90cdf0bbd08f134bc3b49758fb12c493442dba94f9b435a3b33fb7d0289a"} err="failed to get container status \"e9ea90cdf0bbd08f134bc3b49758fb12c493442dba94f9b435a3b33fb7d0289a\": rpc error: code = NotFound desc = could not find container \"e9ea90cdf0bbd08f134bc3b49758fb12c493442dba94f9b435a3b33fb7d0289a\": container with ID starting with e9ea90cdf0bbd08f134bc3b49758fb12c493442dba94f9b435a3b33fb7d0289a not found: ID does not exist"
Sep 18 07:01:51.607558 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.607513    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"]
Sep 18 07:01:51.609567 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:51.609547    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-a7bd58/ipp-route-a7bd58-kserve-5966755cc5-hz9tn"]
Sep 18 07:01:52.542853 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:01:52.542821    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f384570-0092-479f-8afc-a50be435b3bd" path="/var/lib/kubelet/pods/8f384570-0092-479f-8afc-a50be435b3bd/volumes"
Sep 18 07:02:01.413336 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:01.413302    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:02.741697 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:02.741661    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:08.146436 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:08.146401    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:08.206131 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:08.206099    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:08.482363 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:08.482325    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:09.031739 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:09.031709    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:09.671774 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:09.671743    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:10.436274 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:10.436229    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:11.126769 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:11.126736    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:11.837178 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:11.837150    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:19.150980 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.150942    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"]
Sep 18 07:02:19.151470 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.151322    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt" podUID="1537d085-ca41-4bbd-9fa2-9b415ec259a4" containerName="maas-api" containerID="cri-o://300b6320962acfb1801042844dd960cccf607d1a4d9d5f5cecfbd9bf224adef4" gracePeriod=30
Sep 18 07:02:19.180006 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.179975    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:19.247431 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.247392    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:19.321451 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.321424    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:19.377247 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.377203    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:19.394269 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.394245    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"
Sep 18 07:02:19.525030 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.524939    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1537d085-ca41-4bbd-9fa2-9b415ec259a4-maas-api-tls\") pod \"1537d085-ca41-4bbd-9fa2-9b415ec259a4\" (UID: \"1537d085-ca41-4bbd-9fa2-9b415ec259a4\") "
Sep 18 07:02:19.525030 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.525004    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7b6m\" (UniqueName: \"kubernetes.io/projected/1537d085-ca41-4bbd-9fa2-9b415ec259a4-kube-api-access-n7b6m\") pod \"1537d085-ca41-4bbd-9fa2-9b415ec259a4\" (UID: \"1537d085-ca41-4bbd-9fa2-9b415ec259a4\") "
Sep 18 07:02:19.525307 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.525075    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1537d085-ca41-4bbd-9fa2-9b415ec259a4-openshift-service-ca\") pod \"1537d085-ca41-4bbd-9fa2-9b415ec259a4\" (UID: \"1537d085-ca41-4bbd-9fa2-9b415ec259a4\") "
Sep 18 07:02:19.525307 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.525148    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1537d085-ca41-4bbd-9fa2-9b415ec259a4-metrics-certs\") pod \"1537d085-ca41-4bbd-9fa2-9b415ec259a4\" (UID: \"1537d085-ca41-4bbd-9fa2-9b415ec259a4\") "
Sep 18 07:02:19.525811 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.525780    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1537d085-ca41-4bbd-9fa2-9b415ec259a4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1537d085-ca41-4bbd-9fa2-9b415ec259a4" (UID: "1537d085-ca41-4bbd-9fa2-9b415ec259a4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 07:02:19.528470 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.528445    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1537d085-ca41-4bbd-9fa2-9b415ec259a4-kube-api-access-n7b6m" (OuterVolumeSpecName: "kube-api-access-n7b6m") pod "1537d085-ca41-4bbd-9fa2-9b415ec259a4" (UID: "1537d085-ca41-4bbd-9fa2-9b415ec259a4"). InnerVolumeSpecName "kube-api-access-n7b6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 07:02:19.528711 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.528667    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1537d085-ca41-4bbd-9fa2-9b415ec259a4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1537d085-ca41-4bbd-9fa2-9b415ec259a4" (UID: "1537d085-ca41-4bbd-9fa2-9b415ec259a4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 07:02:19.528853 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.528827    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1537d085-ca41-4bbd-9fa2-9b415ec259a4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1537d085-ca41-4bbd-9fa2-9b415ec259a4" (UID: "1537d085-ca41-4bbd-9fa2-9b415ec259a4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 07:02:19.626625 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.626591    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1537d085-ca41-4bbd-9fa2-9b415ec259a4-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:02:19.626625 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.626620    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1537d085-ca41-4bbd-9fa2-9b415ec259a4-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:02:19.626625 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.626629    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1537d085-ca41-4bbd-9fa2-9b415ec259a4-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:02:19.626861 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.626638    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7b6m\" (UniqueName: \"kubernetes.io/projected/1537d085-ca41-4bbd-9fa2-9b415ec259a4-kube-api-access-n7b6m\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:02:19.740095 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.740035    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:19.741272 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.741248    2568 generic.go:356] "Generic (PLEG): container finished" podID="1537d085-ca41-4bbd-9fa2-9b415ec259a4" containerID="300b6320962acfb1801042844dd960cccf607d1a4d9d5f5cecfbd9bf224adef4" exitCode=0
Sep 18 07:02:19.741395 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.741298    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"
Sep 18 07:02:19.741395 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.741302    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt" event={"ID":"1537d085-ca41-4bbd-9fa2-9b415ec259a4","Type":"ContainerDied","Data":"300b6320962acfb1801042844dd960cccf607d1a4d9d5f5cecfbd9bf224adef4"}
Sep 18 07:02:19.741395 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.741342    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt" event={"ID":"1537d085-ca41-4bbd-9fa2-9b415ec259a4","Type":"ContainerDied","Data":"466edb7de792d80a6c4ea0c102634d84bebd99704f779c590f2e63ea11a97dcc"}
Sep 18 07:02:19.741395 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.741363    2568 scope.go:117] "RemoveContainer" containerID="300b6320962acfb1801042844dd960cccf607d1a4d9d5f5cecfbd9bf224adef4"
Sep 18 07:02:19.750979 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.750962    2568 scope.go:117] "RemoveContainer" containerID="300b6320962acfb1801042844dd960cccf607d1a4d9d5f5cecfbd9bf224adef4"
Sep 18 07:02:19.751293 ip-10-0-128-30 kubenswrapper[2568]: E0918 07:02:19.751268    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"300b6320962acfb1801042844dd960cccf607d1a4d9d5f5cecfbd9bf224adef4\": container with ID starting with 300b6320962acfb1801042844dd960cccf607d1a4d9d5f5cecfbd9bf224adef4 not found: ID does not exist" containerID="300b6320962acfb1801042844dd960cccf607d1a4d9d5f5cecfbd9bf224adef4"
Sep 18 07:02:19.751381 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.751303    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"300b6320962acfb1801042844dd960cccf607d1a4d9d5f5cecfbd9bf224adef4"} err="failed to get container status \"300b6320962acfb1801042844dd960cccf607d1a4d9d5f5cecfbd9bf224adef4\": rpc error: code = NotFound desc = could not find container \"300b6320962acfb1801042844dd960cccf607d1a4d9d5f5cecfbd9bf224adef4\": container with ID starting with 300b6320962acfb1801042844dd960cccf607d1a4d9d5f5cecfbd9bf224adef4 not found: ID does not exist"
Sep 18 07:02:19.762300 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.762274    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"]
Sep 18 07:02:19.767796 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.767775    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-1672e6-8587d4474f-q26zt"]
Sep 18 07:02:19.841043 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:19.840866    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:20.407129 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:20.407099    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:20.543973 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:20.543941    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1537d085-ca41-4bbd-9fa2-9b415ec259a4" path="/var/lib/kubelet/pods/1537d085-ca41-4bbd-9fa2-9b415ec259a4/volumes"
Sep 18 07:02:21.126621 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:21.126585    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:22.889846 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:22.889803    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:02:25.344128 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:25.344082    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:25.420610 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:25.420572    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:25.578717 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:25.578686    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"]
Sep 18 07:02:25.578961 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:25.578935    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts" podUID="21b5217f-fb47-4ddd-9416-1a508288f4ac" containerName="istio-proxy" containerID="cri-o://c807873f38efea4c856642cb70ba2b509e799a364c20317e43561bb669c34771" gracePeriod=30
Sep 18 07:02:25.600998 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:25.600921    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:26.051489 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:26.051454    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:26.652438 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:26.652406    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:26.867753 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:26.867720    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:27.647677 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:27.647638    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:28.255330 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:28.255280    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:29.375568 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:29.375532    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:29.425942 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:29.425907    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:30.801067 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.801034    2568 generic.go:356] "Generic (PLEG): container finished" podID="21b5217f-fb47-4ddd-9416-1a508288f4ac" containerID="c807873f38efea4c856642cb70ba2b509e799a364c20317e43561bb669c34771" exitCode=0
Sep 18 07:02:30.801484 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.801112    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts" event={"ID":"21b5217f-fb47-4ddd-9416-1a508288f4ac","Type":"ContainerDied","Data":"c807873f38efea4c856642cb70ba2b509e799a364c20317e43561bb669c34771"}
Sep 18 07:02:30.801484 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.801176    2568 scope.go:117] "RemoveContainer" containerID="82555b00fe926571204d222c65c829ba65a2f409cb9d5549bc965e0ba9801b3f"
Sep 18 07:02:30.827824 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.827807    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 07:02:30.942703 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.942671    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-workload-certs\") pod \"21b5217f-fb47-4ddd-9416-1a508288f4ac\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") "
Sep 18 07:02:30.942900 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.942709    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-data\") pod \"21b5217f-fb47-4ddd-9416-1a508288f4ac\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") "
Sep 18 07:02:30.942900 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.942752    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21b5217f-fb47-4ddd-9416-1a508288f4ac-istiod-ca-cert\") pod \"21b5217f-fb47-4ddd-9416-1a508288f4ac\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") "
Sep 18 07:02:30.942900 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.942814    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsvzh\" (UniqueName: \"kubernetes.io/projected/21b5217f-fb47-4ddd-9416-1a508288f4ac-kube-api-access-zsvzh\") pod \"21b5217f-fb47-4ddd-9416-1a508288f4ac\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") "
Sep 18 07:02:30.942900 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.942850    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-token\") pod \"21b5217f-fb47-4ddd-9416-1a508288f4ac\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") "
Sep 18 07:02:30.942900 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.942885    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-podinfo\") pod \"21b5217f-fb47-4ddd-9416-1a508288f4ac\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") "
Sep 18 07:02:30.943162 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.942913    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-credential-socket\") pod \"21b5217f-fb47-4ddd-9416-1a508288f4ac\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") "
Sep 18 07:02:30.943162 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.942969    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-workload-socket\") pod \"21b5217f-fb47-4ddd-9416-1a508288f4ac\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") "
Sep 18 07:02:30.943162 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.942963    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "21b5217f-fb47-4ddd-9416-1a508288f4ac" (UID: "21b5217f-fb47-4ddd-9416-1a508288f4ac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:02:30.943162 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.943002    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-envoy\") pod \"21b5217f-fb47-4ddd-9416-1a508288f4ac\" (UID: \"21b5217f-fb47-4ddd-9416-1a508288f4ac\") "
Sep 18 07:02:30.943162 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.943046    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-data" (OuterVolumeSpecName: "istio-data") pod "21b5217f-fb47-4ddd-9416-1a508288f4ac" (UID: "21b5217f-fb47-4ddd-9416-1a508288f4ac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:02:30.943445 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.943240    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "21b5217f-fb47-4ddd-9416-1a508288f4ac" (UID: "21b5217f-fb47-4ddd-9416-1a508288f4ac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:02:30.943445 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.943296    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21b5217f-fb47-4ddd-9416-1a508288f4ac-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "21b5217f-fb47-4ddd-9416-1a508288f4ac" (UID: "21b5217f-fb47-4ddd-9416-1a508288f4ac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 07:02:30.943445 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.943304    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "21b5217f-fb47-4ddd-9416-1a508288f4ac" (UID: "21b5217f-fb47-4ddd-9416-1a508288f4ac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:02:30.943601 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.943491    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-data\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:02:30.943601 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.943510    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21b5217f-fb47-4ddd-9416-1a508288f4ac-istiod-ca-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:02:30.943601 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.943524    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-credential-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:02:30.943601 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.943539    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-workload-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:02:30.943601 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.943552    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-workload-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:02:30.945096 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.945073    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "21b5217f-fb47-4ddd-9416-1a508288f4ac" (UID: "21b5217f-fb47-4ddd-9416-1a508288f4ac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 07:02:30.945189 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.945173    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21b5217f-fb47-4ddd-9416-1a508288f4ac-kube-api-access-zsvzh" (OuterVolumeSpecName: "kube-api-access-zsvzh") pod "21b5217f-fb47-4ddd-9416-1a508288f4ac" (UID: "21b5217f-fb47-4ddd-9416-1a508288f4ac"). InnerVolumeSpecName "kube-api-access-zsvzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 07:02:30.945473 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.945455    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "21b5217f-fb47-4ddd-9416-1a508288f4ac" (UID: "21b5217f-fb47-4ddd-9416-1a508288f4ac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:02:30.945473 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:30.945463    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-token" (OuterVolumeSpecName: "istio-token") pod "21b5217f-fb47-4ddd-9416-1a508288f4ac" (UID: "21b5217f-fb47-4ddd-9416-1a508288f4ac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 07:02:31.044807 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:31.044765    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsvzh\" (UniqueName: \"kubernetes.io/projected/21b5217f-fb47-4ddd-9416-1a508288f4ac-kube-api-access-zsvzh\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:02:31.044807 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:31.044793    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-token\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:02:31.044807 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:31.044802    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-podinfo\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:02:31.044807 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:31.044812    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21b5217f-fb47-4ddd-9416-1a508288f4ac-istio-envoy\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:02:31.808074 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:31.808045    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"
Sep 18 07:02:31.808074 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:31.808061    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts" event={"ID":"21b5217f-fb47-4ddd-9416-1a508288f4ac","Type":"ContainerDied","Data":"3495adca49b11dd1327871c5b034a512f966133dad0613e2d75f358a9968718e"}
Sep 18 07:02:31.808607 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:31.808114    2568 scope.go:117] "RemoveContainer" containerID="c807873f38efea4c856642cb70ba2b509e799a364c20317e43561bb669c34771"
Sep 18 07:02:31.841050 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:31.841026    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"]
Sep 18 07:02:31.844384 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:31.844364    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts"]
Sep 18 07:02:32.542844 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:32.542812    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21b5217f-fb47-4ddd-9416-1a508288f4ac" path="/var/lib/kubelet/pods/21b5217f-fb47-4ddd-9416-1a508288f4ac/volumes"
Sep 18 07:02:36.528809 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:36.528774    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ec2077-29828580-c9sgv"]
Sep 18 07:02:36.530360 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:36.530330    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:36.531272 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:36.531245    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ec2077-29828580-c9sgv"]
Sep 18 07:02:36.544529 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:36.544497    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa3965d7-7b89-4bd5-9091-8ec8a6183ef9" path="/var/lib/kubelet/pods/aa3965d7-7b89-4bd5-9091-8ec8a6183ef9/volumes"
Sep 18 07:02:36.596907 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:36.596874    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:36.659012 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:36.658983    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:36.723126 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:36.723093    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:36.780061 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:36.779976    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:37.082584 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:37.082505    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:37.183641 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:37.183595    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:37.755351 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:37.755317    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:39.690375 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:39.690338    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:02:42.847549 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:42.847513    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:42.982205 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:42.982168    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:43.092324 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:43.092294    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"]
Sep 18 07:02:43.092523 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:43.092497    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb" podUID="59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" containerName="istio-proxy" containerID="cri-o://2f572786b3426fcadefc4bb9fb64f1225903b3a6d8aecb977e79affb2f7ff307" gracePeriod=30
Sep 18 07:02:43.114966 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:43.114890    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:43.748969 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:43.748938    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:44.734484 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:44.734450    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:45.347611 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:45.347580    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:48.008310 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.008268    2568 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 07:02:48.008686 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.008352    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb" podUID="59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 07:02:48.342834 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.342804    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 07:02:48.402258 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.402202    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-podinfo\") pod \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") "
Sep 18 07:02:48.402419 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.402274    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-token\") pod \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") "
Sep 18 07:02:48.402419 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.402298    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-data\") pod \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") "
Sep 18 07:02:48.402419 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.402321    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cdkk\" (UniqueName: \"kubernetes.io/projected/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-kube-api-access-2cdkk\") pod \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") "
Sep 18 07:02:48.402419 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.402367    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-workload-socket\") pod \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") "
Sep 18 07:02:48.402629 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.402435    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-workload-certs\") pod \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") "
Sep 18 07:02:48.402629 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.402466    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-envoy\") pod \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") "
Sep 18 07:02:48.402629 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.402509    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istiod-ca-cert\") pod \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") "
Sep 18 07:02:48.402629 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.402548    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-credential-socket\") pod \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\" (UID: \"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf\") "
Sep 18 07:02:48.402629 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.402597    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-data" (OuterVolumeSpecName: "istio-data") pod "59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" (UID: "59f6e498-017b-4b6a-a0ab-5f5ce5b25adf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:02:48.402888 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.402694    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" (UID: "59f6e498-017b-4b6a-a0ab-5f5ce5b25adf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:02:48.402967 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.402944    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-data\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:02:48.403036 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.402971    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-workload-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:02:48.403036 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.402910    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" (UID: "59f6e498-017b-4b6a-a0ab-5f5ce5b25adf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:02:48.403036 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.403003    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" (UID: "59f6e498-017b-4b6a-a0ab-5f5ce5b25adf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:02:48.403036 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.403024    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" (UID: "59f6e498-017b-4b6a-a0ab-5f5ce5b25adf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 07:02:48.404466 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.404440    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-kube-api-access-2cdkk" (OuterVolumeSpecName: "kube-api-access-2cdkk") pod "59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" (UID: "59f6e498-017b-4b6a-a0ab-5f5ce5b25adf"). InnerVolumeSpecName "kube-api-access-2cdkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 07:02:48.404596 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.404571    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" (UID: "59f6e498-017b-4b6a-a0ab-5f5ce5b25adf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 07:02:48.404596 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.404583    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-token" (OuterVolumeSpecName: "istio-token") pod "59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" (UID: "59f6e498-017b-4b6a-a0ab-5f5ce5b25adf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 07:02:48.404702 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.404676    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" (UID: "59f6e498-017b-4b6a-a0ab-5f5ce5b25adf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:02:48.504047 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.504014    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-podinfo\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:02:48.504047 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.504041    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-token\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:02:48.504047 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.504050    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cdkk\" (UniqueName: \"kubernetes.io/projected/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-kube-api-access-2cdkk\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:02:48.504305 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.504061    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-workload-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:02:48.504305 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.504069    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istio-envoy\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:02:48.504305 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.504076    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-istiod-ca-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:02:48.504305 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.504084    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf-credential-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:02:48.902149 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.902114    2568 generic.go:356] "Generic (PLEG): container finished" podID="59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" containerID="2f572786b3426fcadefc4bb9fb64f1225903b3a6d8aecb977e79affb2f7ff307" exitCode=0
Sep 18 07:02:48.902370 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.902168    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"
Sep 18 07:02:48.902370 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.902196    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb" event={"ID":"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf","Type":"ContainerDied","Data":"2f572786b3426fcadefc4bb9fb64f1225903b3a6d8aecb977e79affb2f7ff307"}
Sep 18 07:02:48.902370 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.902269    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb" event={"ID":"59f6e498-017b-4b6a-a0ab-5f5ce5b25adf","Type":"ContainerDied","Data":"14c0b0aa5649f4c4703fd82a5582b78aef7b2cf76a5086afda0a968b7336371b"}
Sep 18 07:02:48.902370 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.902292    2568 scope.go:117] "RemoveContainer" containerID="2f572786b3426fcadefc4bb9fb64f1225903b3a6d8aecb977e79affb2f7ff307"
Sep 18 07:02:48.911345 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.911326    2568 scope.go:117] "RemoveContainer" containerID="2aacb02d2c8fe9153c5c42c4b858a2b1795192e5db01e612a204448bd85f87c3"
Sep 18 07:02:48.919243 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.919206    2568 scope.go:117] "RemoveContainer" containerID="2f572786b3426fcadefc4bb9fb64f1225903b3a6d8aecb977e79affb2f7ff307"
Sep 18 07:02:48.919521 ip-10-0-128-30 kubenswrapper[2568]: E0918 07:02:48.919501    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f572786b3426fcadefc4bb9fb64f1225903b3a6d8aecb977e79affb2f7ff307\": container with ID starting with 2f572786b3426fcadefc4bb9fb64f1225903b3a6d8aecb977e79affb2f7ff307 not found: ID does not exist" containerID="2f572786b3426fcadefc4bb9fb64f1225903b3a6d8aecb977e79affb2f7ff307"
Sep 18 07:02:48.919585 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.919530    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f572786b3426fcadefc4bb9fb64f1225903b3a6d8aecb977e79affb2f7ff307"} err="failed to get container status \"2f572786b3426fcadefc4bb9fb64f1225903b3a6d8aecb977e79affb2f7ff307\": rpc error: code = NotFound desc = could not find container \"2f572786b3426fcadefc4bb9fb64f1225903b3a6d8aecb977e79affb2f7ff307\": container with ID starting with 2f572786b3426fcadefc4bb9fb64f1225903b3a6d8aecb977e79affb2f7ff307 not found: ID does not exist"
Sep 18 07:02:48.919585 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.919551    2568 scope.go:117] "RemoveContainer" containerID="2aacb02d2c8fe9153c5c42c4b858a2b1795192e5db01e612a204448bd85f87c3"
Sep 18 07:02:48.919703 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.919596    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"]
Sep 18 07:02:48.919842 ip-10-0-128-30 kubenswrapper[2568]: E0918 07:02:48.919827    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2aacb02d2c8fe9153c5c42c4b858a2b1795192e5db01e612a204448bd85f87c3\": container with ID starting with 2aacb02d2c8fe9153c5c42c4b858a2b1795192e5db01e612a204448bd85f87c3 not found: ID does not exist" containerID="2aacb02d2c8fe9153c5c42c4b858a2b1795192e5db01e612a204448bd85f87c3"
Sep 18 07:02:48.919887 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.919850    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aacb02d2c8fe9153c5c42c4b858a2b1795192e5db01e612a204448bd85f87c3"} err="failed to get container status \"2aacb02d2c8fe9153c5c42c4b858a2b1795192e5db01e612a204448bd85f87c3\": rpc error: code = NotFound desc = could not find container \"2aacb02d2c8fe9153c5c42c4b858a2b1795192e5db01e612a204448bd85f87c3\": container with ID starting with 2aacb02d2c8fe9153c5c42c4b858a2b1795192e5db01e612a204448bd85f87c3 not found: ID does not exist"
Sep 18 07:02:48.922812 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:48.922782    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb"]
Sep 18 07:02:50.543423 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:50.543390    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" path="/var/lib/kubelet/pods/59f6e498-017b-4b6a-a0ab-5f5ce5b25adf/volumes"
Sep 18 07:02:54.020791 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:54.020751    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-8fdd5b-29828580-mmprx"]
Sep 18 07:02:54.022050 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:54.022023    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:54.023636 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:54.023608    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-8fdd5b-29828580-mmprx"]
Sep 18 07:02:54.543038 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:54.543000    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32e453be-f3ab-4ef9-9cb7-b886ec15a3c6" path="/var/lib/kubelet/pods/32e453be-f3ab-4ef9-9cb7-b886ec15a3c6/volumes"
Sep 18 07:02:55.269463 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:55.269429    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:02:56.586653 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:02:56.586617    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:03:00.294800 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:00.294771    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:03:00.349197 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:00.349168    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:03:00.534849 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:00.534814    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"]
Sep 18 07:03:00.535104 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:00.535074    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s" podUID="01a52fd4-e07e-407c-9192-1467b42b6d78" containerName="istio-proxy" containerID="cri-o://dcf16968054977490b4f62edd85464b3823a35b8087bdf1d21cddea3a0fbf0f8" gracePeriod=30
Sep 18 07:03:00.556187 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:00.556114    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:03:05.784053 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.784026    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 07:03:05.872646 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.872572    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lm9qr\" (UniqueName: \"kubernetes.io/projected/01a52fd4-e07e-407c-9192-1467b42b6d78-kube-api-access-lm9qr\") pod \"01a52fd4-e07e-407c-9192-1467b42b6d78\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") "
Sep 18 07:03:05.872646 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.872627    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-credential-socket\") pod \"01a52fd4-e07e-407c-9192-1467b42b6d78\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") "
Sep 18 07:03:05.872845 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.872766    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-workload-socket\") pod \"01a52fd4-e07e-407c-9192-1467b42b6d78\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") "
Sep 18 07:03:05.872845 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.872802    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-envoy\") pod \"01a52fd4-e07e-407c-9192-1467b42b6d78\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") "
Sep 18 07:03:05.872845 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.872824    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-token\") pod \"01a52fd4-e07e-407c-9192-1467b42b6d78\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") "
Sep 18 07:03:05.872845 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.872842    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-workload-certs\") pod \"01a52fd4-e07e-407c-9192-1467b42b6d78\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") "
Sep 18 07:03:05.873055 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.872880    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "01a52fd4-e07e-407c-9192-1467b42b6d78" (UID: "01a52fd4-e07e-407c-9192-1467b42b6d78"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:03:05.873055 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.872907    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-podinfo\") pod \"01a52fd4-e07e-407c-9192-1467b42b6d78\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") "
Sep 18 07:03:05.873055 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.872930    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a52fd4-e07e-407c-9192-1467b42b6d78-istiod-ca-cert\") pod \"01a52fd4-e07e-407c-9192-1467b42b6d78\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") "
Sep 18 07:03:05.873055 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.872968    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-data\") pod \"01a52fd4-e07e-407c-9192-1467b42b6d78\" (UID: \"01a52fd4-e07e-407c-9192-1467b42b6d78\") "
Sep 18 07:03:05.873055 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.873006    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "01a52fd4-e07e-407c-9192-1467b42b6d78" (UID: "01a52fd4-e07e-407c-9192-1467b42b6d78"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:03:05.873369 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.873133    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "01a52fd4-e07e-407c-9192-1467b42b6d78" (UID: "01a52fd4-e07e-407c-9192-1467b42b6d78"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:03:05.873431 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.873379    2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-credential-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:03:05.873431 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.873383    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-data" (OuterVolumeSpecName: "istio-data") pod "01a52fd4-e07e-407c-9192-1467b42b6d78" (UID: "01a52fd4-e07e-407c-9192-1467b42b6d78"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:03:05.873431 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.873399    2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-workload-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:03:05.873431 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.873409    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01a52fd4-e07e-407c-9192-1467b42b6d78-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "01a52fd4-e07e-407c-9192-1467b42b6d78" (UID: "01a52fd4-e07e-407c-9192-1467b42b6d78"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 07:03:05.873640 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.873416    2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-workload-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:03:05.875070 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.875039    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "01a52fd4-e07e-407c-9192-1467b42b6d78" (UID: "01a52fd4-e07e-407c-9192-1467b42b6d78"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 07:03:05.875070 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.875053    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-token" (OuterVolumeSpecName: "istio-token") pod "01a52fd4-e07e-407c-9192-1467b42b6d78" (UID: "01a52fd4-e07e-407c-9192-1467b42b6d78"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 07:03:05.875246 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.875092    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01a52fd4-e07e-407c-9192-1467b42b6d78-kube-api-access-lm9qr" (OuterVolumeSpecName: "kube-api-access-lm9qr") pod "01a52fd4-e07e-407c-9192-1467b42b6d78" (UID: "01a52fd4-e07e-407c-9192-1467b42b6d78"). InnerVolumeSpecName "kube-api-access-lm9qr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 07:03:05.875246 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.875162    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "01a52fd4-e07e-407c-9192-1467b42b6d78" (UID: "01a52fd4-e07e-407c-9192-1467b42b6d78"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:03:05.889405 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.889381    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:03:05.974547 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.974511    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lm9qr\" (UniqueName: \"kubernetes.io/projected/01a52fd4-e07e-407c-9192-1467b42b6d78-kube-api-access-lm9qr\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:03:05.974547 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.974543    2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-envoy\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:03:05.974731 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.974559    2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-token\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:03:05.974731 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.974571    2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-podinfo\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:03:05.974731 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.974598    2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01a52fd4-e07e-407c-9192-1467b42b6d78-istiod-ca-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:03:05.974731 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.974611    2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01a52fd4-e07e-407c-9192-1467b42b6d78-istio-data\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:03:05.986063 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.986033    2568 generic.go:356] "Generic (PLEG): container finished" podID="01a52fd4-e07e-407c-9192-1467b42b6d78" containerID="dcf16968054977490b4f62edd85464b3823a35b8087bdf1d21cddea3a0fbf0f8" exitCode=0
Sep 18 07:03:05.986186 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.986079    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s" event={"ID":"01a52fd4-e07e-407c-9192-1467b42b6d78","Type":"ContainerDied","Data":"dcf16968054977490b4f62edd85464b3823a35b8087bdf1d21cddea3a0fbf0f8"}
Sep 18 07:03:05.986186 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.986119    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s" event={"ID":"01a52fd4-e07e-407c-9192-1467b42b6d78","Type":"ContainerDied","Data":"453fd65e94977de399a86903dcd62359858e4c6f99d223776dba7b1f9b7c88b8"}
Sep 18 07:03:05.986186 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.986138    2568 scope.go:117] "RemoveContainer" containerID="dcf16968054977490b4f62edd85464b3823a35b8087bdf1d21cddea3a0fbf0f8"
Sep 18 07:03:05.986186 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.986082    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"
Sep 18 07:03:05.995136 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:05.995116    2568 scope.go:117] "RemoveContainer" containerID="3b7e66f7a963c226cbbbea081d42efee06c033634da85dc1f654db247ab5280b"
Sep 18 07:03:06.002989 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:06.002974    2568 scope.go:117] "RemoveContainer" containerID="dcf16968054977490b4f62edd85464b3823a35b8087bdf1d21cddea3a0fbf0f8"
Sep 18 07:03:06.003255 ip-10-0-128-30 kubenswrapper[2568]: E0918 07:03:06.003213    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcf16968054977490b4f62edd85464b3823a35b8087bdf1d21cddea3a0fbf0f8\": container with ID starting with dcf16968054977490b4f62edd85464b3823a35b8087bdf1d21cddea3a0fbf0f8 not found: ID does not exist" containerID="dcf16968054977490b4f62edd85464b3823a35b8087bdf1d21cddea3a0fbf0f8"
Sep 18 07:03:06.003370 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:06.003258    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcf16968054977490b4f62edd85464b3823a35b8087bdf1d21cddea3a0fbf0f8"} err="failed to get container status \"dcf16968054977490b4f62edd85464b3823a35b8087bdf1d21cddea3a0fbf0f8\": rpc error: code = NotFound desc = could not find container \"dcf16968054977490b4f62edd85464b3823a35b8087bdf1d21cddea3a0fbf0f8\": container with ID starting with dcf16968054977490b4f62edd85464b3823a35b8087bdf1d21cddea3a0fbf0f8 not found: ID does not exist"
Sep 18 07:03:06.003370 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:06.003275    2568 scope.go:117] "RemoveContainer" containerID="3b7e66f7a963c226cbbbea081d42efee06c033634da85dc1f654db247ab5280b"
Sep 18 07:03:06.003518 ip-10-0-128-30 kubenswrapper[2568]: E0918 07:03:06.003501    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b7e66f7a963c226cbbbea081d42efee06c033634da85dc1f654db247ab5280b\": container with ID starting with 3b7e66f7a963c226cbbbea081d42efee06c033634da85dc1f654db247ab5280b not found: ID does not exist" containerID="3b7e66f7a963c226cbbbea081d42efee06c033634da85dc1f654db247ab5280b"
Sep 18 07:03:06.003556 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:06.003523    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b7e66f7a963c226cbbbea081d42efee06c033634da85dc1f654db247ab5280b"} err="failed to get container status \"3b7e66f7a963c226cbbbea081d42efee06c033634da85dc1f654db247ab5280b\": rpc error: code = NotFound desc = could not find container \"3b7e66f7a963c226cbbbea081d42efee06c033634da85dc1f654db247ab5280b\": container with ID starting with 3b7e66f7a963c226cbbbea081d42efee06c033634da85dc1f654db247ab5280b not found: ID does not exist"
Sep 18 07:03:06.006979 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:06.006961    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"]
Sep 18 07:03:06.010317 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:06.010297    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s"]
Sep 18 07:03:06.554591 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:06.554556    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01a52fd4-e07e-407c-9192-1467b42b6d78" path="/var/lib/kubelet/pods/01a52fd4-e07e-407c-9192-1467b42b6d78/volumes"
Sep 18 07:03:32.923387 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:32.923344    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:03:56.144666 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:56.144631    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:03:56.195021 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:03:56.194988    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:04:26.684287 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:04:26.684247    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:04:27.003821 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:04:27.003742    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:04:28.495334 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:04:28.495305    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:05:06.081734 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:06.081695    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:05:14.380853 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:14.380819    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:05:15.031633 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:15.031600    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:05:22.384226 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:22.384191    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:05:40.487075 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:40.486979    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:05:46.986748 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:46.986718    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:05:48.135645 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:48.135602    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq"]
Sep 18 07:05:48.136144 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:48.135828    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq" podUID="de65d394-4e57-415d-8b9c-42dfb499258c" containerName="manager" containerID="cri-o://695e2697c31563983d95fa57bd26489b75b52e89cb1d0202ddcdd176a31acdcb" gracePeriod=10
Sep 18 07:05:50.486665 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:50.486636    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq"
Sep 18 07:05:50.600681 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:50.600594    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sngm\" (UniqueName: \"kubernetes.io/projected/de65d394-4e57-415d-8b9c-42dfb499258c-kube-api-access-5sngm\") pod \"de65d394-4e57-415d-8b9c-42dfb499258c\" (UID: \"de65d394-4e57-415d-8b9c-42dfb499258c\") "
Sep 18 07:05:50.600681 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:50.600652    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/de65d394-4e57-415d-8b9c-42dfb499258c-extensions-socket-volume\") pod \"de65d394-4e57-415d-8b9c-42dfb499258c\" (UID: \"de65d394-4e57-415d-8b9c-42dfb499258c\") "
Sep 18 07:05:50.601038 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:50.601005    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de65d394-4e57-415d-8b9c-42dfb499258c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "de65d394-4e57-415d-8b9c-42dfb499258c" (UID: "de65d394-4e57-415d-8b9c-42dfb499258c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:05:50.602665 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:50.602632    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de65d394-4e57-415d-8b9c-42dfb499258c-kube-api-access-5sngm" (OuterVolumeSpecName: "kube-api-access-5sngm") pod "de65d394-4e57-415d-8b9c-42dfb499258c" (UID: "de65d394-4e57-415d-8b9c-42dfb499258c"). InnerVolumeSpecName "kube-api-access-5sngm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 07:05:50.701865 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:50.701829    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sngm\" (UniqueName: \"kubernetes.io/projected/de65d394-4e57-415d-8b9c-42dfb499258c-kube-api-access-5sngm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:05:50.701865 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:50.701857    2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/de65d394-4e57-415d-8b9c-42dfb499258c-extensions-socket-volume\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:05:50.803681 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:50.803648    2568 generic.go:356] "Generic (PLEG): container finished" podID="de65d394-4e57-415d-8b9c-42dfb499258c" containerID="695e2697c31563983d95fa57bd26489b75b52e89cb1d0202ddcdd176a31acdcb" exitCode=0
Sep 18 07:05:50.803841 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:50.803692    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq"
Sep 18 07:05:50.803841 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:50.803700    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq" event={"ID":"de65d394-4e57-415d-8b9c-42dfb499258c","Type":"ContainerDied","Data":"695e2697c31563983d95fa57bd26489b75b52e89cb1d0202ddcdd176a31acdcb"}
Sep 18 07:05:50.803841 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:50.803744    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq" event={"ID":"de65d394-4e57-415d-8b9c-42dfb499258c","Type":"ContainerDied","Data":"e3e009fe21177100b64c227b675c8d7cc2f753531bb8e6aee448c0353b5de002"}
Sep 18 07:05:50.803841 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:50.803766    2568 scope.go:117] "RemoveContainer" containerID="695e2697c31563983d95fa57bd26489b75b52e89cb1d0202ddcdd176a31acdcb"
Sep 18 07:05:50.813301 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:50.813289    2568 scope.go:117] "RemoveContainer" containerID="695e2697c31563983d95fa57bd26489b75b52e89cb1d0202ddcdd176a31acdcb"
Sep 18 07:05:50.813541 ip-10-0-128-30 kubenswrapper[2568]: E0918 07:05:50.813522    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"695e2697c31563983d95fa57bd26489b75b52e89cb1d0202ddcdd176a31acdcb\": container with ID starting with 695e2697c31563983d95fa57bd26489b75b52e89cb1d0202ddcdd176a31acdcb not found: ID does not exist" containerID="695e2697c31563983d95fa57bd26489b75b52e89cb1d0202ddcdd176a31acdcb"
Sep 18 07:05:50.813593 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:50.813551    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"695e2697c31563983d95fa57bd26489b75b52e89cb1d0202ddcdd176a31acdcb"} err="failed to get container status \"695e2697c31563983d95fa57bd26489b75b52e89cb1d0202ddcdd176a31acdcb\": rpc error: code = NotFound desc = could not find container \"695e2697c31563983d95fa57bd26489b75b52e89cb1d0202ddcdd176a31acdcb\": container with ID starting with 695e2697c31563983d95fa57bd26489b75b52e89cb1d0202ddcdd176a31acdcb not found: ID does not exist"
Sep 18 07:05:50.824546 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:50.824525    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq"]
Sep 18 07:05:50.828306 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:50.828282    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-zqxbq"]
Sep 18 07:05:52.542736 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:52.542700    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de65d394-4e57-415d-8b9c-42dfb499258c" path="/var/lib/kubelet/pods/de65d394-4e57-415d-8b9c-42dfb499258c/volumes"
Sep 18 07:05:57.982985 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:57.982950    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:05:58.517949 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:58.517918    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:05:58.618320 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:05:58.618284    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:06:16.958123 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:16.958091    2568 scope.go:117] "RemoveContainer" containerID="164d40bfa4bb0a5d3f956efcf9282f4c2d3231aeb8d00fd3c12ecf76634e9db4"
Sep 18 07:06:16.966002 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:16.965992    2568 scope.go:117] "RemoveContainer" containerID="1625768dcbbcc678e52c61f1de83047f529ea95caccd505edf77aa9ade6f5d9c"
Sep 18 07:06:48.830096 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830057    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rcqw9"]
Sep 18 07:06:48.830605 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830445    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01a52fd4-e07e-407c-9192-1467b42b6d78" containerName="istio-proxy"
Sep 18 07:06:48.830605 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830457    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="01a52fd4-e07e-407c-9192-1467b42b6d78" containerName="istio-proxy"
Sep 18 07:06:48.830605 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830466    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa3965d7-7b89-4bd5-9091-8ec8a6183ef9" containerName="cleanup"
Sep 18 07:06:48.830605 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830471    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa3965d7-7b89-4bd5-9091-8ec8a6183ef9" containerName="cleanup"
Sep 18 07:06:48.830605 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830478    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" containerName="istio-proxy"
Sep 18 07:06:48.830605 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830484    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" containerName="istio-proxy"
Sep 18 07:06:48.830605 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830494    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32e453be-f3ab-4ef9-9cb7-b886ec15a3c6" containerName="cleanup"
Sep 18 07:06:48.830605 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830500    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="32e453be-f3ab-4ef9-9cb7-b886ec15a3c6" containerName="cleanup"
Sep 18 07:06:48.830605 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830508    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" containerName="istio-proxy"
Sep 18 07:06:48.830605 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830514    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" containerName="istio-proxy"
Sep 18 07:06:48.830605 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830521    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f384570-0092-479f-8afc-a50be435b3bd" containerName="main"
Sep 18 07:06:48.830605 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830526    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f384570-0092-479f-8afc-a50be435b3bd" containerName="main"
Sep 18 07:06:48.830605 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830533    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1537d085-ca41-4bbd-9fa2-9b415ec259a4" containerName="maas-api"
Sep 18 07:06:48.830605 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830538    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1537d085-ca41-4bbd-9fa2-9b415ec259a4" containerName="maas-api"
Sep 18 07:06:48.830605 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830546    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21b5217f-fb47-4ddd-9416-1a508288f4ac" containerName="istio-proxy"
Sep 18 07:06:48.830605 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830551    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="21b5217f-fb47-4ddd-9416-1a508288f4ac" containerName="istio-proxy"
Sep 18 07:06:48.830605 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830557    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de65d394-4e57-415d-8b9c-42dfb499258c" containerName="manager"
Sep 18 07:06:48.830605 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830562    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="de65d394-4e57-415d-8b9c-42dfb499258c" containerName="manager"
Sep 18 07:06:48.830605 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830569    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21b5217f-fb47-4ddd-9416-1a508288f4ac" containerName="istio-proxy"
Sep 18 07:06:48.830605 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830574    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="21b5217f-fb47-4ddd-9416-1a508288f4ac" containerName="istio-proxy"
Sep 18 07:06:48.831207 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830628    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="32e453be-f3ab-4ef9-9cb7-b886ec15a3c6" containerName="cleanup"
Sep 18 07:06:48.831207 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830637    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" containerName="istio-proxy"
Sep 18 07:06:48.831207 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830646    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="21b5217f-fb47-4ddd-9416-1a508288f4ac" containerName="istio-proxy"
Sep 18 07:06:48.831207 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830651    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="59f6e498-017b-4b6a-a0ab-5f5ce5b25adf" containerName="istio-proxy"
Sep 18 07:06:48.831207 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830656    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="01a52fd4-e07e-407c-9192-1467b42b6d78" containerName="istio-proxy"
Sep 18 07:06:48.831207 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830661    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa3965d7-7b89-4bd5-9091-8ec8a6183ef9" containerName="cleanup"
Sep 18 07:06:48.831207 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830666    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f384570-0092-479f-8afc-a50be435b3bd" containerName="main"
Sep 18 07:06:48.831207 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830671    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1537d085-ca41-4bbd-9fa2-9b415ec259a4" containerName="maas-api"
Sep 18 07:06:48.831207 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830676    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="01a52fd4-e07e-407c-9192-1467b42b6d78" containerName="istio-proxy"
Sep 18 07:06:48.831207 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830682    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="21b5217f-fb47-4ddd-9416-1a508288f4ac" containerName="istio-proxy"
Sep 18 07:06:48.831207 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.830688    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="de65d394-4e57-415d-8b9c-42dfb499258c" containerName="manager"
Sep 18 07:06:48.833722 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.833705    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rcqw9"
Sep 18 07:06:48.836418 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.836392    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-fhtgh\""
Sep 18 07:06:48.838494 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.838468    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rcqw9"]
Sep 18 07:06:48.911729 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.911693    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl9qj\" (UniqueName: \"kubernetes.io/projected/e1bd7958-e146-46d5-963e-3deb0ef63943-kube-api-access-kl9qj\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rcqw9\" (UID: \"e1bd7958-e146-46d5-963e-3deb0ef63943\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rcqw9"
Sep 18 07:06:48.911729 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:48.911732    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1bd7958-e146-46d5-963e-3deb0ef63943-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rcqw9\" (UID: \"e1bd7958-e146-46d5-963e-3deb0ef63943\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rcqw9"
Sep 18 07:06:49.012944 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:49.012907    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl9qj\" (UniqueName: \"kubernetes.io/projected/e1bd7958-e146-46d5-963e-3deb0ef63943-kube-api-access-kl9qj\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rcqw9\" (UID: \"e1bd7958-e146-46d5-963e-3deb0ef63943\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rcqw9"
Sep 18 07:06:49.012944 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:49.012949    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1bd7958-e146-46d5-963e-3deb0ef63943-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rcqw9\" (UID: \"e1bd7958-e146-46d5-963e-3deb0ef63943\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rcqw9"
Sep 18 07:06:49.013361 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:49.013343    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1bd7958-e146-46d5-963e-3deb0ef63943-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rcqw9\" (UID: \"e1bd7958-e146-46d5-963e-3deb0ef63943\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rcqw9"
Sep 18 07:06:49.021971 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:49.021951    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl9qj\" (UniqueName: \"kubernetes.io/projected/e1bd7958-e146-46d5-963e-3deb0ef63943-kube-api-access-kl9qj\") pod \"kuadrant-operator-controller-manager-5769f99d9f-rcqw9\" (UID: \"e1bd7958-e146-46d5-963e-3deb0ef63943\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rcqw9"
Sep 18 07:06:49.145200 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:49.145118    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rcqw9"
Sep 18 07:06:49.418947 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:49.418920    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rcqw9"]
Sep 18 07:06:49.421380 ip-10-0-128-30 kubenswrapper[2568]: W0918 07:06:49.421350    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1bd7958_e146_46d5_963e_3deb0ef63943.slice/crio-da157d83faf751e15be8a61613724a5e05a5ff96a4bc7126f291c3bd5cf8c990 WatchSource:0}: Error finding container da157d83faf751e15be8a61613724a5e05a5ff96a4bc7126f291c3bd5cf8c990: Status 404 returned error can't find the container with id da157d83faf751e15be8a61613724a5e05a5ff96a4bc7126f291c3bd5cf8c990
Sep 18 07:06:50.080955 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:50.080918    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rcqw9" event={"ID":"e1bd7958-e146-46d5-963e-3deb0ef63943","Type":"ContainerStarted","Data":"546397a5219e4f5329c00ac59b5e0b690cfb9d99a4cf5e8ae4e6abfb8f77f3c6"}
Sep 18 07:06:50.081433 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:50.080975    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rcqw9" event={"ID":"e1bd7958-e146-46d5-963e-3deb0ef63943","Type":"ContainerStarted","Data":"da157d83faf751e15be8a61613724a5e05a5ff96a4bc7126f291c3bd5cf8c990"}
Sep 18 07:06:50.081433 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:50.081022    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rcqw9"
Sep 18 07:06:50.101112 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:06:50.101057    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rcqw9" podStartSLOduration=2.101041629 podStartE2EDuration="2.101041629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 07:06:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 07:06:50.099070584 +0000 UTC m=+2675.780598327" watchObservedRunningTime="2026-09-18 07:06:50.101041629 +0000 UTC m=+2675.782569372"
Sep 18 07:07:01.088378 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:01.088295    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-rcqw9"
Sep 18 07:07:16.391333 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:16.391299    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:07:22.580396 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:22.580361    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:07:34.020346 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:34.020300    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:07:52.098070 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.098034    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"]
Sep 18 07:07:52.098664 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.098642    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01a52fd4-e07e-407c-9192-1467b42b6d78" containerName="istio-proxy"
Sep 18 07:07:52.098715 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.098668    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="01a52fd4-e07e-407c-9192-1467b42b6d78" containerName="istio-proxy"
Sep 18 07:07:52.101924 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.101907    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:07:52.104621 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.104594    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-08dfaed8-kserve-self-signed-certs\""
Sep 18 07:07:52.111565 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.111539    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"]
Sep 18 07:07:52.124036 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.124007    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:07:52.193187 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.193154    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn66t\" (UniqueName: \"kubernetes.io/projected/40ef59e8-de42-4685-b3c8-7fb5275de291-kube-api-access-dn66t\") pod \"e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") " pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:07:52.193383 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.193200    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-model-cache\") pod \"e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") " pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:07:52.193383 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.193270    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-tmp-dir\") pod \"e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") " pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:07:52.193383 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.193325    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-dshm\") pod \"e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") " pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:07:52.193383 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.193351    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-home\") pod \"e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") " pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:07:52.193558 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.193410    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40ef59e8-de42-4685-b3c8-7fb5275de291-tls-certs\") pod \"e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") " pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:07:52.294267 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.294228    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn66t\" (UniqueName: \"kubernetes.io/projected/40ef59e8-de42-4685-b3c8-7fb5275de291-kube-api-access-dn66t\") pod \"e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") " pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:07:52.294476 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.294293    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-model-cache\") pod \"e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") " pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:07:52.294476 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.294321    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-tmp-dir\") pod \"e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") " pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:07:52.294476 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.294347    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-dshm\") pod \"e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") " pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:07:52.294476 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.294374    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-home\") pod \"e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") " pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:07:52.294476 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.294426    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40ef59e8-de42-4685-b3c8-7fb5275de291-tls-certs\") pod \"e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") " pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:07:52.294772 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.294744    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-model-cache\") pod \"e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") " pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:07:52.294838 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.294766    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-tmp-dir\") pod \"e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") " pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:07:52.294900 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.294860    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-home\") pod \"e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") " pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:07:52.296585 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.296557    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-dshm\") pod \"e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") " pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:07:52.296828 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.296812    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40ef59e8-de42-4685-b3c8-7fb5275de291-tls-certs\") pod \"e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") " pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:07:52.302102 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.302079    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn66t\" (UniqueName: \"kubernetes.io/projected/40ef59e8-de42-4685-b3c8-7fb5275de291-kube-api-access-dn66t\") pod \"e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") " pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:07:52.412612 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.412588    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:07:52.687363 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:52.687307    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"]
Sep 18 07:07:52.689712 ip-10-0-128-30 kubenswrapper[2568]: W0918 07:07:52.689684    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40ef59e8_de42_4685_b3c8_7fb5275de291.slice/crio-538d1efb8f87d674caaff5f661e92ca5dcbdbebab1cf368c432b7de148a87af7 WatchSource:0}: Error finding container 538d1efb8f87d674caaff5f661e92ca5dcbdbebab1cf368c432b7de148a87af7: Status 404 returned error can't find the container with id 538d1efb8f87d674caaff5f661e92ca5dcbdbebab1cf368c432b7de148a87af7
Sep 18 07:07:53.379412 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:53.379373    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw" event={"ID":"40ef59e8-de42-4685-b3c8-7fb5275de291","Type":"ContainerStarted","Data":"b7f3089fdf8c077e073365065a31cdb7ce307664cef69542c39e70aff436582e"}
Sep 18 07:07:53.379894 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:53.379422    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw" event={"ID":"40ef59e8-de42-4685-b3c8-7fb5275de291","Type":"ContainerStarted","Data":"538d1efb8f87d674caaff5f661e92ca5dcbdbebab1cf368c432b7de148a87af7"}
Sep 18 07:07:53.396644 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:53.396596    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw" podStartSLOduration=1.3965805869999999 podStartE2EDuration="1.396580587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 07:07:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 07:07:53.395099911 +0000 UTC m=+2739.076627684" watchObservedRunningTime="2026-09-18 07:07:53.396580587 +0000 UTC m=+2739.078108329"
Sep 18 07:07:54.587699 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:54.587664    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:07:58.489375 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.489345    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"]
Sep 18 07:07:58.492947 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.492928    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:07:58.495564 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.495538    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-08dfaed8-kserve-self-signed-certs\""
Sep 18 07:07:58.501297 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.501274    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"]
Sep 18 07:07:58.554883 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.554804    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-home\") pod \"e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") " pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:07:58.554883 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.554866    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-model-cache\") pod \"e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") " pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:07:58.555124 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.554892    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v86lb\" (UniqueName: \"kubernetes.io/projected/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-kube-api-access-v86lb\") pod \"e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") " pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:07:58.555124 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.555037    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-tls-certs\") pod \"e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") " pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:07:58.555251 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.555168    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-dshm\") pod \"e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") " pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:07:58.555251 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.555197    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-tmp-dir\") pod \"e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") " pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:07:58.604753 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.604724    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:07:58.656247 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.656188    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-dshm\") pod \"e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") " pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:07:58.656247 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.656243    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-tmp-dir\") pod \"e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") " pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:07:58.656497 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.656354    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-home\") pod \"e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") " pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:07:58.656497 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.656411    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-model-cache\") pod \"e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") " pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:07:58.656497 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.656437    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v86lb\" (UniqueName: \"kubernetes.io/projected/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-kube-api-access-v86lb\") pod \"e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") " pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:07:58.656663 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.656558    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-tls-certs\") pod \"e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") " pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:07:58.656663 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.656596    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-tmp-dir\") pod \"e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") " pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:07:58.656777 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.656722    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-home\") pod \"e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") " pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:07:58.656987 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.656960    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-model-cache\") pod \"e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") " pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:07:58.658467 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.658446    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-dshm\") pod \"e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") " pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:07:58.658820 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.658804    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-tls-certs\") pod \"e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") " pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:07:58.664186 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.664165    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v86lb\" (UniqueName: \"kubernetes.io/projected/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-kube-api-access-v86lb\") pod \"e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") " pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:07:58.803002 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:58.802918    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:07:59.073038 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:59.073012    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"]
Sep 18 07:07:59.075960 ip-10-0-128-30 kubenswrapper[2568]: W0918 07:07:59.075932    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf2ba7b6c_4a88_4b2e_af1a_410bb7167b20.slice/crio-79eb0b6031c97c3af3b4915b0eec8cf4770da4ea94ce1a36f6145828c25f909b WatchSource:0}: Error finding container 79eb0b6031c97c3af3b4915b0eec8cf4770da4ea94ce1a36f6145828c25f909b: Status 404 returned error can't find the container with id 79eb0b6031c97c3af3b4915b0eec8cf4770da4ea94ce1a36f6145828c25f909b
Sep 18 07:07:59.410018 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:59.409981    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69" event={"ID":"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20","Type":"ContainerStarted","Data":"4ee7574d771a51213c17a6ea07970a534e75b15135434185b555b23c3e491427"}
Sep 18 07:07:59.410195 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:59.410031    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69" event={"ID":"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20","Type":"ContainerStarted","Data":"79eb0b6031c97c3af3b4915b0eec8cf4770da4ea94ce1a36f6145828c25f909b"}
Sep 18 07:07:59.427820 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:07:59.427775    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69" podStartSLOduration=1.4277608960000001 podStartE2EDuration="1.427760896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 07:07:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 07:07:59.425976478 +0000 UTC m=+2745.107504221" watchObservedRunningTime="2026-09-18 07:07:59.427760896 +0000 UTC m=+2745.109288641"
Sep 18 07:08:02.413635 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:02.413603    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:08:02.414076 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:02.413653    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:08:02.426286 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:02.426263    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:08:02.438239 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:02.438201    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:08:02.504888 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:02.504847    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:08:08.803522 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:08.803491    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:08:08.803939 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:08.803730    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:08:08.815947 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:08.815926    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:08:09.473431 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:09.473401    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:08:26.000852 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:26.000816    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"]
Sep 18 07:08:26.020615 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:26.020583    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:08:27.080360 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:27.080274    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"]
Sep 18 07:08:27.101568 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:27.101539    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:08:29.303334 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:29.303294    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:08:39.090693 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:39.090649    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:08:41.020691 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.020651    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw" podUID="40ef59e8-de42-4685-b3c8-7fb5275de291" containerName="main" containerID="cri-o://b7f3089fdf8c077e073365065a31cdb7ce307664cef69542c39e70aff436582e" gracePeriod=45
Sep 18 07:08:41.271777 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.271715    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:08:41.356016 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.355985    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn66t\" (UniqueName: \"kubernetes.io/projected/40ef59e8-de42-4685-b3c8-7fb5275de291-kube-api-access-dn66t\") pod \"40ef59e8-de42-4685-b3c8-7fb5275de291\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") "
Sep 18 07:08:41.356242 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.356036    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-tmp-dir\") pod \"40ef59e8-de42-4685-b3c8-7fb5275de291\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") "
Sep 18 07:08:41.356242 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.356061    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-dshm\") pod \"40ef59e8-de42-4685-b3c8-7fb5275de291\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") "
Sep 18 07:08:41.356242 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.356120    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-home\") pod \"40ef59e8-de42-4685-b3c8-7fb5275de291\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") "
Sep 18 07:08:41.356242 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.356156    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-model-cache\") pod \"40ef59e8-de42-4685-b3c8-7fb5275de291\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") "
Sep 18 07:08:41.356474 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.356296    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40ef59e8-de42-4685-b3c8-7fb5275de291-tls-certs\") pod \"40ef59e8-de42-4685-b3c8-7fb5275de291\" (UID: \"40ef59e8-de42-4685-b3c8-7fb5275de291\") "
Sep 18 07:08:41.356474 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.356384    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "40ef59e8-de42-4685-b3c8-7fb5275de291" (UID: "40ef59e8-de42-4685-b3c8-7fb5275de291"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:08:41.356474 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.356396    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-home" (OuterVolumeSpecName: "home") pod "40ef59e8-de42-4685-b3c8-7fb5275de291" (UID: "40ef59e8-de42-4685-b3c8-7fb5275de291"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:08:41.356474 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.356452    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-model-cache" (OuterVolumeSpecName: "model-cache") pod "40ef59e8-de42-4685-b3c8-7fb5275de291" (UID: "40ef59e8-de42-4685-b3c8-7fb5275de291"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:08:41.356690 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.356670    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:08:41.356745 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.356696    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:08:41.356745 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.356711    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:08:41.358384 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.358357    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-dshm" (OuterVolumeSpecName: "dshm") pod "40ef59e8-de42-4685-b3c8-7fb5275de291" (UID: "40ef59e8-de42-4685-b3c8-7fb5275de291"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:08:41.358466 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.358440    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40ef59e8-de42-4685-b3c8-7fb5275de291-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "40ef59e8-de42-4685-b3c8-7fb5275de291" (UID: "40ef59e8-de42-4685-b3c8-7fb5275de291"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 07:08:41.358507 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.358471    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40ef59e8-de42-4685-b3c8-7fb5275de291-kube-api-access-dn66t" (OuterVolumeSpecName: "kube-api-access-dn66t") pod "40ef59e8-de42-4685-b3c8-7fb5275de291" (UID: "40ef59e8-de42-4685-b3c8-7fb5275de291"). InnerVolumeSpecName "kube-api-access-dn66t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 07:08:41.457465 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.457433    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/40ef59e8-de42-4685-b3c8-7fb5275de291-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:08:41.457465 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.457461    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn66t\" (UniqueName: \"kubernetes.io/projected/40ef59e8-de42-4685-b3c8-7fb5275de291-kube-api-access-dn66t\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:08:41.457465 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.457471    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/40ef59e8-de42-4685-b3c8-7fb5275de291-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:08:41.627954 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.627874    2568 generic.go:356] "Generic (PLEG): container finished" podID="40ef59e8-de42-4685-b3c8-7fb5275de291" containerID="b7f3089fdf8c077e073365065a31cdb7ce307664cef69542c39e70aff436582e" exitCode=0
Sep 18 07:08:41.627954 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.627914    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"
Sep 18 07:08:41.628135 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.627960    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw" event={"ID":"40ef59e8-de42-4685-b3c8-7fb5275de291","Type":"ContainerDied","Data":"b7f3089fdf8c077e073365065a31cdb7ce307664cef69542c39e70aff436582e"}
Sep 18 07:08:41.628135 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.628000    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw" event={"ID":"40ef59e8-de42-4685-b3c8-7fb5275de291","Type":"ContainerDied","Data":"538d1efb8f87d674caaff5f661e92ca5dcbdbebab1cf368c432b7de148a87af7"}
Sep 18 07:08:41.628135 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.628018    2568 scope.go:117] "RemoveContainer" containerID="b7f3089fdf8c077e073365065a31cdb7ce307664cef69542c39e70aff436582e"
Sep 18 07:08:41.637312 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.637293    2568 scope.go:117] "RemoveContainer" containerID="b7f3089fdf8c077e073365065a31cdb7ce307664cef69542c39e70aff436582e"
Sep 18 07:08:41.637594 ip-10-0-128-30 kubenswrapper[2568]: E0918 07:08:41.637576    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7f3089fdf8c077e073365065a31cdb7ce307664cef69542c39e70aff436582e\": container with ID starting with b7f3089fdf8c077e073365065a31cdb7ce307664cef69542c39e70aff436582e not found: ID does not exist" containerID="b7f3089fdf8c077e073365065a31cdb7ce307664cef69542c39e70aff436582e"
Sep 18 07:08:41.637677 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.637601    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7f3089fdf8c077e073365065a31cdb7ce307664cef69542c39e70aff436582e"} err="failed to get container status \"b7f3089fdf8c077e073365065a31cdb7ce307664cef69542c39e70aff436582e\": rpc error: code = NotFound desc = could not find container \"b7f3089fdf8c077e073365065a31cdb7ce307664cef69542c39e70aff436582e\": container with ID starting with b7f3089fdf8c077e073365065a31cdb7ce307664cef69542c39e70aff436582e not found: ID does not exist"
Sep 18 07:08:41.648191 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.648169    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"]
Sep 18 07:08:41.651980 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:41.651958    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-08dfaed8-kserve-79954d96c5-2plpw"]
Sep 18 07:08:42.099715 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.099657    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69" podUID="f2ba7b6c-4a88-4b2e-af1a-410bb7167b20" containerName="main" containerID="cri-o://4ee7574d771a51213c17a6ea07970a534e75b15135434185b555b23c3e491427" gracePeriod=45
Sep 18 07:08:42.364276 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.364205    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:08:42.467437 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.467410    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-tmp-dir\") pod \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") "
Sep 18 07:08:42.467626 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.467448    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-dshm\") pod \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") "
Sep 18 07:08:42.467626 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.467496    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-model-cache\") pod \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") "
Sep 18 07:08:42.467626 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.467515    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-home\") pod \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") "
Sep 18 07:08:42.467626 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.467557    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v86lb\" (UniqueName: \"kubernetes.io/projected/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-kube-api-access-v86lb\") pod \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") "
Sep 18 07:08:42.467626 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.467604    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-tls-certs\") pod \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\" (UID: \"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20\") "
Sep 18 07:08:42.467906 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.467715    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f2ba7b6c-4a88-4b2e-af1a-410bb7167b20" (UID: "f2ba7b6c-4a88-4b2e-af1a-410bb7167b20"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:08:42.467906 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.467773    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-model-cache" (OuterVolumeSpecName: "model-cache") pod "f2ba7b6c-4a88-4b2e-af1a-410bb7167b20" (UID: "f2ba7b6c-4a88-4b2e-af1a-410bb7167b20"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:08:42.467906 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.467803    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-home" (OuterVolumeSpecName: "home") pod "f2ba7b6c-4a88-4b2e-af1a-410bb7167b20" (UID: "f2ba7b6c-4a88-4b2e-af1a-410bb7167b20"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:08:42.468013 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.467910    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:08:42.468013 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.467928    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:08:42.468013 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.467937    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:08:42.469674 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.469646    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-dshm" (OuterVolumeSpecName: "dshm") pod "f2ba7b6c-4a88-4b2e-af1a-410bb7167b20" (UID: "f2ba7b6c-4a88-4b2e-af1a-410bb7167b20"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 07:08:42.469790 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.469680    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f2ba7b6c-4a88-4b2e-af1a-410bb7167b20" (UID: "f2ba7b6c-4a88-4b2e-af1a-410bb7167b20"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 07:08:42.469790 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.469744    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-kube-api-access-v86lb" (OuterVolumeSpecName: "kube-api-access-v86lb") pod "f2ba7b6c-4a88-4b2e-af1a-410bb7167b20" (UID: "f2ba7b6c-4a88-4b2e-af1a-410bb7167b20"). InnerVolumeSpecName "kube-api-access-v86lb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 07:08:42.542656 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.542623    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40ef59e8-de42-4685-b3c8-7fb5275de291" path="/var/lib/kubelet/pods/40ef59e8-de42-4685-b3c8-7fb5275de291/volumes"
Sep 18 07:08:42.569347 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.569321    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v86lb\" (UniqueName: \"kubernetes.io/projected/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-kube-api-access-v86lb\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:08:42.569347 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.569345    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:08:42.569501 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.569355    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:08:42.635785 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.635709    2568 generic.go:356] "Generic (PLEG): container finished" podID="f2ba7b6c-4a88-4b2e-af1a-410bb7167b20" containerID="4ee7574d771a51213c17a6ea07970a534e75b15135434185b555b23c3e491427" exitCode=0
Sep 18 07:08:42.635785 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.635756    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"
Sep 18 07:08:42.635956 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.635795    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69" event={"ID":"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20","Type":"ContainerDied","Data":"4ee7574d771a51213c17a6ea07970a534e75b15135434185b555b23c3e491427"}
Sep 18 07:08:42.635956 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.635844    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69" event={"ID":"f2ba7b6c-4a88-4b2e-af1a-410bb7167b20","Type":"ContainerDied","Data":"79eb0b6031c97c3af3b4915b0eec8cf4770da4ea94ce1a36f6145828c25f909b"}
Sep 18 07:08:42.635956 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.635861    2568 scope.go:117] "RemoveContainer" containerID="4ee7574d771a51213c17a6ea07970a534e75b15135434185b555b23c3e491427"
Sep 18 07:08:42.648092 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.648070    2568 scope.go:117] "RemoveContainer" containerID="4ee7574d771a51213c17a6ea07970a534e75b15135434185b555b23c3e491427"
Sep 18 07:08:42.648420 ip-10-0-128-30 kubenswrapper[2568]: E0918 07:08:42.648401    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ee7574d771a51213c17a6ea07970a534e75b15135434185b555b23c3e491427\": container with ID starting with 4ee7574d771a51213c17a6ea07970a534e75b15135434185b555b23c3e491427 not found: ID does not exist" containerID="4ee7574d771a51213c17a6ea07970a534e75b15135434185b555b23c3e491427"
Sep 18 07:08:42.648494 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.648431    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ee7574d771a51213c17a6ea07970a534e75b15135434185b555b23c3e491427"} err="failed to get container status \"4ee7574d771a51213c17a6ea07970a534e75b15135434185b555b23c3e491427\": rpc error: code = NotFound desc = could not find container \"4ee7574d771a51213c17a6ea07970a534e75b15135434185b555b23c3e491427\": container with ID starting with 4ee7574d771a51213c17a6ea07970a534e75b15135434185b555b23c3e491427 not found: ID does not exist"
Sep 18 07:08:42.663008 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.662980    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"]
Sep 18 07:08:42.667670 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:42.667650    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-08dfaed8-kserve-d97f99c8c-r7h69"]
Sep 18 07:08:44.543004 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:44.542971    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2ba7b6c-4a88-4b2e-af1a-410bb7167b20" path="/var/lib/kubelet/pods/f2ba7b6c-4a88-4b2e-af1a-410bb7167b20/volumes"
Sep 18 07:08:50.696387 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:50.691846    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:08:50.934004 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:50.933970    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"]
Sep 18 07:08:50.934371 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:50.934358    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2ba7b6c-4a88-4b2e-af1a-410bb7167b20" containerName="main"
Sep 18 07:08:50.934371 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:50.934372    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2ba7b6c-4a88-4b2e-af1a-410bb7167b20" containerName="main"
Sep 18 07:08:50.934468 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:50.934396    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40ef59e8-de42-4685-b3c8-7fb5275de291" containerName="main"
Sep 18 07:08:50.934468 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:50.934401    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="40ef59e8-de42-4685-b3c8-7fb5275de291" containerName="main"
Sep 18 07:08:50.934468 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:50.934462    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="40ef59e8-de42-4685-b3c8-7fb5275de291" containerName="main"
Sep 18 07:08:50.934612 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:50.934472    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2ba7b6c-4a88-4b2e-af1a-410bb7167b20" containerName="main"
Sep 18 07:08:50.937899 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:50.937881    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"
Sep 18 07:08:50.942879 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:50.942858    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-mxmzc\""
Sep 18 07:08:50.942992 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:50.942860    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 18 07:08:50.943714 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:50.943697    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 07:08:50.946136 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:50.946116    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 18 07:08:50.946514 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:50.946468    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 18 07:08:50.951569 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:50.951549    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"]
Sep 18 07:08:51.043715 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:51.043676    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14e54543-00a6-4a77-87a3-21be77cf9925-metrics-certs\") pod \"maas-api-dc699bbb9-rnlh7\" (UID: \"14e54543-00a6-4a77-87a3-21be77cf9925\") " pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"
Sep 18 07:08:51.043905 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:51.043730    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnrhd\" (UniqueName: \"kubernetes.io/projected/14e54543-00a6-4a77-87a3-21be77cf9925-kube-api-access-jnrhd\") pod \"maas-api-dc699bbb9-rnlh7\" (UID: \"14e54543-00a6-4a77-87a3-21be77cf9925\") " pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"
Sep 18 07:08:51.043905 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:51.043808    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14e54543-00a6-4a77-87a3-21be77cf9925-openshift-service-ca\") pod \"maas-api-dc699bbb9-rnlh7\" (UID: \"14e54543-00a6-4a77-87a3-21be77cf9925\") " pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"
Sep 18 07:08:51.043905 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:51.043896    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14e54543-00a6-4a77-87a3-21be77cf9925-maas-api-tls\") pod \"maas-api-dc699bbb9-rnlh7\" (UID: \"14e54543-00a6-4a77-87a3-21be77cf9925\") " pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"
Sep 18 07:08:51.144810 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:51.144769    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14e54543-00a6-4a77-87a3-21be77cf9925-maas-api-tls\") pod \"maas-api-dc699bbb9-rnlh7\" (UID: \"14e54543-00a6-4a77-87a3-21be77cf9925\") " pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"
Sep 18 07:08:51.145014 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:51.144848    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14e54543-00a6-4a77-87a3-21be77cf9925-metrics-certs\") pod \"maas-api-dc699bbb9-rnlh7\" (UID: \"14e54543-00a6-4a77-87a3-21be77cf9925\") " pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"
Sep 18 07:08:51.145014 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:51.144872    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnrhd\" (UniqueName: \"kubernetes.io/projected/14e54543-00a6-4a77-87a3-21be77cf9925-kube-api-access-jnrhd\") pod \"maas-api-dc699bbb9-rnlh7\" (UID: \"14e54543-00a6-4a77-87a3-21be77cf9925\") " pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"
Sep 18 07:08:51.145014 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:51.144916    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14e54543-00a6-4a77-87a3-21be77cf9925-openshift-service-ca\") pod \"maas-api-dc699bbb9-rnlh7\" (UID: \"14e54543-00a6-4a77-87a3-21be77cf9925\") " pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"
Sep 18 07:08:51.145654 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:51.145626    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14e54543-00a6-4a77-87a3-21be77cf9925-openshift-service-ca\") pod \"maas-api-dc699bbb9-rnlh7\" (UID: \"14e54543-00a6-4a77-87a3-21be77cf9925\") " pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"
Sep 18 07:08:51.147283 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:51.147256    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14e54543-00a6-4a77-87a3-21be77cf9925-maas-api-tls\") pod \"maas-api-dc699bbb9-rnlh7\" (UID: \"14e54543-00a6-4a77-87a3-21be77cf9925\") " pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"
Sep 18 07:08:51.147433 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:51.147405    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14e54543-00a6-4a77-87a3-21be77cf9925-metrics-certs\") pod \"maas-api-dc699bbb9-rnlh7\" (UID: \"14e54543-00a6-4a77-87a3-21be77cf9925\") " pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"
Sep 18 07:08:51.154128 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:51.154101    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnrhd\" (UniqueName: \"kubernetes.io/projected/14e54543-00a6-4a77-87a3-21be77cf9925-kube-api-access-jnrhd\") pod \"maas-api-dc699bbb9-rnlh7\" (UID: \"14e54543-00a6-4a77-87a3-21be77cf9925\") " pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"
Sep 18 07:08:51.247904 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:51.247816    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"
Sep 18 07:08:51.521027 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:51.520999    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"]
Sep 18 07:08:51.522251 ip-10-0-128-30 kubenswrapper[2568]: W0918 07:08:51.522197    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14e54543_00a6_4a77_87a3_21be77cf9925.slice/crio-753a7c03d2f54748a15b205e00b8e521f5a399c3db5c6007a9a8f22c3f4c4e7f WatchSource:0}: Error finding container 753a7c03d2f54748a15b205e00b8e521f5a399c3db5c6007a9a8f22c3f4c4e7f: Status 404 returned error can't find the container with id 753a7c03d2f54748a15b205e00b8e521f5a399c3db5c6007a9a8f22c3f4c4e7f
Sep 18 07:08:51.523784 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:51.523762    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 07:08:51.689161 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:51.689125    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7" event={"ID":"14e54543-00a6-4a77-87a3-21be77cf9925","Type":"ContainerStarted","Data":"753a7c03d2f54748a15b205e00b8e521f5a399c3db5c6007a9a8f22c3f4c4e7f"}
Sep 18 07:08:52.695037 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:52.695004    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7" event={"ID":"14e54543-00a6-4a77-87a3-21be77cf9925","Type":"ContainerStarted","Data":"6235d3fc1c356935dd3d21525cd17b4b9361f5f236503340a5276b26ed366c0e"}
Sep 18 07:08:52.695503 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:52.695118    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"
Sep 18 07:08:52.714598 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:52.714556    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7" podStartSLOduration=2.12225742 podStartE2EDuration="2.714541933s" totalImagesPullingTime="592.284513ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 07:08:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 07:08:52.712377665 +0000 UTC m=+2798.393905409" watchObservedRunningTime="2026-09-18 07:08:52.714541933 +0000 UTC m=+2798.396069677"
Sep 18 07:08:57.096679 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:57.096646    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:08:58.705836 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:58.705809    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"
Sep 18 07:08:59.181737 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:59.181703    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8bdb6f45c-cxk4v"]
Sep 18 07:08:59.185047 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:59.185031    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8bdb6f45c-cxk4v"
Sep 18 07:08:59.198742 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:59.198713    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8bdb6f45c-cxk4v"]
Sep 18 07:08:59.326241 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:59.326187    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e566c29-d083-486b-8a92-93b8d25cb6e5-openshift-service-ca\") pod \"maas-api-8bdb6f45c-cxk4v\" (UID: \"0e566c29-d083-486b-8a92-93b8d25cb6e5\") " pod="odh-ai-gateway-infra/maas-api-8bdb6f45c-cxk4v"
Sep 18 07:08:59.326415 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:59.326249    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e566c29-d083-486b-8a92-93b8d25cb6e5-maas-api-tls\") pod \"maas-api-8bdb6f45c-cxk4v\" (UID: \"0e566c29-d083-486b-8a92-93b8d25cb6e5\") " pod="odh-ai-gateway-infra/maas-api-8bdb6f45c-cxk4v"
Sep 18 07:08:59.326415 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:59.326318    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e566c29-d083-486b-8a92-93b8d25cb6e5-metrics-certs\") pod \"maas-api-8bdb6f45c-cxk4v\" (UID: \"0e566c29-d083-486b-8a92-93b8d25cb6e5\") " pod="odh-ai-gateway-infra/maas-api-8bdb6f45c-cxk4v"
Sep 18 07:08:59.326415 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:59.326350    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pkkw\" (UniqueName: \"kubernetes.io/projected/0e566c29-d083-486b-8a92-93b8d25cb6e5-kube-api-access-8pkkw\") pod \"maas-api-8bdb6f45c-cxk4v\" (UID: \"0e566c29-d083-486b-8a92-93b8d25cb6e5\") " pod="odh-ai-gateway-infra/maas-api-8bdb6f45c-cxk4v"
Sep 18 07:08:59.427092 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:59.427059    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e566c29-d083-486b-8a92-93b8d25cb6e5-metrics-certs\") pod \"maas-api-8bdb6f45c-cxk4v\" (UID: \"0e566c29-d083-486b-8a92-93b8d25cb6e5\") " pod="odh-ai-gateway-infra/maas-api-8bdb6f45c-cxk4v"
Sep 18 07:08:59.427092 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:59.427094    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pkkw\" (UniqueName: \"kubernetes.io/projected/0e566c29-d083-486b-8a92-93b8d25cb6e5-kube-api-access-8pkkw\") pod \"maas-api-8bdb6f45c-cxk4v\" (UID: \"0e566c29-d083-486b-8a92-93b8d25cb6e5\") " pod="odh-ai-gateway-infra/maas-api-8bdb6f45c-cxk4v"
Sep 18 07:08:59.427332 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:59.427150    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e566c29-d083-486b-8a92-93b8d25cb6e5-openshift-service-ca\") pod \"maas-api-8bdb6f45c-cxk4v\" (UID: \"0e566c29-d083-486b-8a92-93b8d25cb6e5\") " pod="odh-ai-gateway-infra/maas-api-8bdb6f45c-cxk4v"
Sep 18 07:08:59.427332 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:59.427168    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e566c29-d083-486b-8a92-93b8d25cb6e5-maas-api-tls\") pod \"maas-api-8bdb6f45c-cxk4v\" (UID: \"0e566c29-d083-486b-8a92-93b8d25cb6e5\") " pod="odh-ai-gateway-infra/maas-api-8bdb6f45c-cxk4v"
Sep 18 07:08:59.427883 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:59.427859    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e566c29-d083-486b-8a92-93b8d25cb6e5-openshift-service-ca\") pod \"maas-api-8bdb6f45c-cxk4v\" (UID: \"0e566c29-d083-486b-8a92-93b8d25cb6e5\") " pod="odh-ai-gateway-infra/maas-api-8bdb6f45c-cxk4v"
Sep 18 07:08:59.429525 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:59.429503    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e566c29-d083-486b-8a92-93b8d25cb6e5-metrics-certs\") pod \"maas-api-8bdb6f45c-cxk4v\" (UID: \"0e566c29-d083-486b-8a92-93b8d25cb6e5\") " pod="odh-ai-gateway-infra/maas-api-8bdb6f45c-cxk4v"
Sep 18 07:08:59.429659 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:59.429603    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e566c29-d083-486b-8a92-93b8d25cb6e5-maas-api-tls\") pod \"maas-api-8bdb6f45c-cxk4v\" (UID: \"0e566c29-d083-486b-8a92-93b8d25cb6e5\") " pod="odh-ai-gateway-infra/maas-api-8bdb6f45c-cxk4v"
Sep 18 07:08:59.436901 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:59.436831    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pkkw\" (UniqueName: \"kubernetes.io/projected/0e566c29-d083-486b-8a92-93b8d25cb6e5-kube-api-access-8pkkw\") pod \"maas-api-8bdb6f45c-cxk4v\" (UID: \"0e566c29-d083-486b-8a92-93b8d25cb6e5\") " pod="odh-ai-gateway-infra/maas-api-8bdb6f45c-cxk4v"
Sep 18 07:08:59.494565 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:59.494534    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8bdb6f45c-cxk4v"
Sep 18 07:08:59.774981 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:08:59.774955    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8bdb6f45c-cxk4v"]
Sep 18 07:08:59.777391 ip-10-0-128-30 kubenswrapper[2568]: W0918 07:08:59.777358    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e566c29_d083_486b_8a92_93b8d25cb6e5.slice/crio-f46161e849587c2521768b7bb3f3fc0975b3ecf6e28ec8842abd73ac30f87730 WatchSource:0}: Error finding container f46161e849587c2521768b7bb3f3fc0975b3ecf6e28ec8842abd73ac30f87730: Status 404 returned error can't find the container with id f46161e849587c2521768b7bb3f3fc0975b3ecf6e28ec8842abd73ac30f87730
Sep 18 07:09:00.733647 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:00.733616    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8bdb6f45c-cxk4v" event={"ID":"0e566c29-d083-486b-8a92-93b8d25cb6e5","Type":"ContainerStarted","Data":"be63ab99d1257b47bf12b6a92ab8923cfdba6f4e730354853fa8b316a66fbc72"}
Sep 18 07:09:00.733824 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:00.733673    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8bdb6f45c-cxk4v" event={"ID":"0e566c29-d083-486b-8a92-93b8d25cb6e5","Type":"ContainerStarted","Data":"f46161e849587c2521768b7bb3f3fc0975b3ecf6e28ec8842abd73ac30f87730"}
Sep 18 07:09:00.733824 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:00.733702    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8bdb6f45c-cxk4v"
Sep 18 07:09:00.754704 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:00.754657    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8bdb6f45c-cxk4v" podStartSLOduration=1.256241518 podStartE2EDuration="1.75464074s" totalImagesPullingTime="498.399222ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 07:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 07:09:00.753098268 +0000 UTC m=+2806.434626012" watchObservedRunningTime="2026-09-18 07:09:00.75464074 +0000 UTC m=+2806.436168487"
Sep 18 07:09:03.114939 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:03.114901    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:09:06.752062 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:06.752037    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8bdb6f45c-cxk4v"
Sep 18 07:09:06.847527 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:06.847497    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"]
Sep 18 07:09:06.847726 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:06.847699    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7" podUID="14e54543-00a6-4a77-87a3-21be77cf9925" containerName="maas-api" containerID="cri-o://6235d3fc1c356935dd3d21525cd17b4b9361f5f236503340a5276b26ed366c0e" gracePeriod=30
Sep 18 07:09:07.090724 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:07.090703    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"
Sep 18 07:09:07.092788 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:07.092771    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnrhd\" (UniqueName: \"kubernetes.io/projected/14e54543-00a6-4a77-87a3-21be77cf9925-kube-api-access-jnrhd\") pod \"14e54543-00a6-4a77-87a3-21be77cf9925\" (UID: \"14e54543-00a6-4a77-87a3-21be77cf9925\") "
Sep 18 07:09:07.092849 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:07.092824    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14e54543-00a6-4a77-87a3-21be77cf9925-openshift-service-ca\") pod \"14e54543-00a6-4a77-87a3-21be77cf9925\" (UID: \"14e54543-00a6-4a77-87a3-21be77cf9925\") "
Sep 18 07:09:07.092883 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:07.092856    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14e54543-00a6-4a77-87a3-21be77cf9925-metrics-certs\") pod \"14e54543-00a6-4a77-87a3-21be77cf9925\" (UID: \"14e54543-00a6-4a77-87a3-21be77cf9925\") "
Sep 18 07:09:07.092883 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:07.092878    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14e54543-00a6-4a77-87a3-21be77cf9925-maas-api-tls\") pod \"14e54543-00a6-4a77-87a3-21be77cf9925\" (UID: \"14e54543-00a6-4a77-87a3-21be77cf9925\") "
Sep 18 07:09:07.093230 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:07.093198    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14e54543-00a6-4a77-87a3-21be77cf9925-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "14e54543-00a6-4a77-87a3-21be77cf9925" (UID: "14e54543-00a6-4a77-87a3-21be77cf9925"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 07:09:07.094916 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:07.094891    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14e54543-00a6-4a77-87a3-21be77cf9925-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "14e54543-00a6-4a77-87a3-21be77cf9925" (UID: "14e54543-00a6-4a77-87a3-21be77cf9925"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 07:09:07.095004 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:07.094932    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14e54543-00a6-4a77-87a3-21be77cf9925-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "14e54543-00a6-4a77-87a3-21be77cf9925" (UID: "14e54543-00a6-4a77-87a3-21be77cf9925"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 07:09:07.095134 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:07.095119    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14e54543-00a6-4a77-87a3-21be77cf9925-kube-api-access-jnrhd" (OuterVolumeSpecName: "kube-api-access-jnrhd") pod "14e54543-00a6-4a77-87a3-21be77cf9925" (UID: "14e54543-00a6-4a77-87a3-21be77cf9925"). InnerVolumeSpecName "kube-api-access-jnrhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 07:09:07.194584 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:07.194542    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnrhd\" (UniqueName: \"kubernetes.io/projected/14e54543-00a6-4a77-87a3-21be77cf9925-kube-api-access-jnrhd\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:09:07.194584 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:07.194583    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/14e54543-00a6-4a77-87a3-21be77cf9925-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:09:07.194864 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:07.194600    2568 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14e54543-00a6-4a77-87a3-21be77cf9925-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:09:07.194864 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:07.194622    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/14e54543-00a6-4a77-87a3-21be77cf9925-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 18 07:09:07.776441 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:07.776403    2568 generic.go:356] "Generic (PLEG): container finished" podID="14e54543-00a6-4a77-87a3-21be77cf9925" containerID="6235d3fc1c356935dd3d21525cd17b4b9361f5f236503340a5276b26ed366c0e" exitCode=0
Sep 18 07:09:07.776870 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:07.776452    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7" event={"ID":"14e54543-00a6-4a77-87a3-21be77cf9925","Type":"ContainerDied","Data":"6235d3fc1c356935dd3d21525cd17b4b9361f5f236503340a5276b26ed366c0e"}
Sep 18 07:09:07.776870 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:07.776460    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"
Sep 18 07:09:07.776870 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:07.776503    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7" event={"ID":"14e54543-00a6-4a77-87a3-21be77cf9925","Type":"ContainerDied","Data":"753a7c03d2f54748a15b205e00b8e521f5a399c3db5c6007a9a8f22c3f4c4e7f"}
Sep 18 07:09:07.776870 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:07.776528    2568 scope.go:117] "RemoveContainer" containerID="6235d3fc1c356935dd3d21525cd17b4b9361f5f236503340a5276b26ed366c0e"
Sep 18 07:09:07.785969 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:07.785954    2568 scope.go:117] "RemoveContainer" containerID="6235d3fc1c356935dd3d21525cd17b4b9361f5f236503340a5276b26ed366c0e"
Sep 18 07:09:07.786240 ip-10-0-128-30 kubenswrapper[2568]: E0918 07:09:07.786203    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6235d3fc1c356935dd3d21525cd17b4b9361f5f236503340a5276b26ed366c0e\": container with ID starting with 6235d3fc1c356935dd3d21525cd17b4b9361f5f236503340a5276b26ed366c0e not found: ID does not exist" containerID="6235d3fc1c356935dd3d21525cd17b4b9361f5f236503340a5276b26ed366c0e"
Sep 18 07:09:07.786343 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:07.786248    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6235d3fc1c356935dd3d21525cd17b4b9361f5f236503340a5276b26ed366c0e"} err="failed to get container status \"6235d3fc1c356935dd3d21525cd17b4b9361f5f236503340a5276b26ed366c0e\": rpc error: code = NotFound desc = could not find container \"6235d3fc1c356935dd3d21525cd17b4b9361f5f236503340a5276b26ed366c0e\": container with ID starting with 6235d3fc1c356935dd3d21525cd17b4b9361f5f236503340a5276b26ed366c0e not found: ID does not exist"
Sep 18 07:09:07.809912 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:07.809884    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"]
Sep 18 07:09:07.829277 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:07.829254    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-dc699bbb9-rnlh7"]
Sep 18 07:09:08.544974 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:08.544937    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14e54543-00a6-4a77-87a3-21be77cf9925" path="/var/lib/kubelet/pods/14e54543-00a6-4a77-87a3-21be77cf9925/volumes"
Sep 18 07:09:09.131581 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:09.131547    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:09:16.210816 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:16.210779    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-sczjl"]
Sep 18 07:09:24.857978 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:24.857946    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-jz5zd_d377a298-99f3-4042-98c8-303150a24782/manager/0.log"
Sep 18 07:09:24.986125 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:24.986095    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-5dl2q_24e331ac-1ca6-484e-a96a-2603f3f51732/manager/0.log"
Sep 18 07:09:25.250134 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:25.250103    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-nfc2b_8947c3c0-f29d-44a8-a085-6877f589af98/server/0.log"
Sep 18 07:09:25.778163 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:25.778135    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fd575c64f-xv5qk_54caad4e-0c6a-43f8-bdc3-eabffaaf7c19/copy-manifests/0.log"
Sep 18 07:09:25.787228 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:25.787200    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fd575c64f-xv5qk_54caad4e-0c6a-43f8-bdc3-eabffaaf7c19/manager/0.log"
Sep 18 07:09:26.261626 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:26.261600    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-8bdb6f45c-cxk4v_0e566c29-d083-486b-8a92-93b8d25cb6e5/maas-api/0.log"
Sep 18 07:09:27.572281 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:27.572252    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-q78cp_3c03039b-93f8-489e-8675-9af35729e24b/manager/0.log"
Sep 18 07:09:27.956161 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:27.956133    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-rcqw9_e1bd7958-e146-46d5-963e-3deb0ef63943/manager/0.log"
Sep 18 07:09:28.086269 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:28.086244    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-sczjl_fa87303d-4f34-4bb3-9cf6-9c53882f33eb/limitador/0.log"
Sep 18 07:09:28.211865 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:28.211791    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-t2r5j_c9b510d5-fc33-4bb6-a8ea-a6e7574305b3/manager/0.log"
Sep 18 07:09:28.835356 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:28.835325    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bbd68dc59-6grwd_415ce0db-109b-4944-b4aa-2ed7733a8b0e/kube-auth-proxy/0.log"
Sep 18 07:09:29.073877 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:29.073849    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-f2pj4_0484017e-ee10-41d4-b810-b3a80f6382da/istio-proxy/1.log"
Sep 18 07:09:30.360950 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:30.360917    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5jn88_b0111315-8eb7-45db-8729-e99cb59b4df6/main/0.log"
Sep 18 07:09:30.539367 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:30.539334    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-h5j6g_8868048f-b3d5-4257-97ca-027cd06d1fc2/main/0.log"
Sep 18 07:09:30.659213 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:30.659185    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-dr59q_d0ef703e-fddd-4851-956f-ab20938918eb/main/0.log"
Sep 18 07:09:36.335386 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:36.335350    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:09:36.544144 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:36.544109    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log" limit=1024
Sep 18 07:09:39.898816 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:39.898791    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-n8fkx_1a0016b5-3d41-4c0c-afd6-bf60b5e69237/global-pull-secret-syncer/0.log"
Sep 18 07:09:40.010169 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:40.010145    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2ffp7_d1f8c4d2-31e1-4cca-b763-dc559fa12937/konnectivity-agent/0.log"
Sep 18 07:09:40.197724 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:40.197646    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-30.ec2.internal_aab83480badcc6c24454042cf62b9928/haproxy/0.log"
Sep 18 07:09:43.778016 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:43.777987    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-q78cp_3c03039b-93f8-489e-8675-9af35729e24b/manager/0.log"
Sep 18 07:09:43.902685 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:43.902651    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-rcqw9_e1bd7958-e146-46d5-963e-3deb0ef63943/manager/0.log"
Sep 18 07:09:43.960684 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:43.960658    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-sczjl_fa87303d-4f34-4bb3-9cf6-9c53882f33eb/limitador/0.log"
Sep 18 07:09:44.024244 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:44.024207    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-t2r5j_c9b510d5-fc33-4bb6-a8ea-a6e7574305b3/manager/0.log"
Sep 18 07:09:45.731019 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:45.730990    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_623e7f44-b4e5-43df-99ed-b09621ccddf5/alertmanager/0.log"
Sep 18 07:09:45.758164 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:45.758145    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_623e7f44-b4e5-43df-99ed-b09621ccddf5/config-reloader/0.log"
Sep 18 07:09:45.793663 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:45.793639    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_623e7f44-b4e5-43df-99ed-b09621ccddf5/kube-rbac-proxy-web/0.log"
Sep 18 07:09:45.836825 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:45.836804    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_623e7f44-b4e5-43df-99ed-b09621ccddf5/kube-rbac-proxy/0.log"
Sep 18 07:09:45.871751 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:45.871730    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_623e7f44-b4e5-43df-99ed-b09621ccddf5/kube-rbac-proxy-metric/0.log"
Sep 18 07:09:45.906328 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:45.906303    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_623e7f44-b4e5-43df-99ed-b09621ccddf5/prom-label-proxy/0.log"
Sep 18 07:09:45.935341 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:45.935314    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_623e7f44-b4e5-43df-99ed-b09621ccddf5/init-config-reloader/0.log"
Sep 18 07:09:46.469970 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:46.469939    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x4vkx_56815ee5-968b-4af9-95ad-3cf971e1e720/node-exporter/0.log"
Sep 18 07:09:46.494996 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:46.494969    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x4vkx_56815ee5-968b-4af9-95ad-3cf971e1e720/kube-rbac-proxy/0.log"
Sep 18 07:09:46.532863 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:46.532841    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x4vkx_56815ee5-968b-4af9-95ad-3cf971e1e720/init-textfile/0.log"
Sep 18 07:09:47.159315 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:47.159293    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-886668cc9-5qcpz_94e58ef4-4b8d-438b-91a3-f4b0be69234a/telemeter-client/0.log"
Sep 18 07:09:47.197965 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:47.197943    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-886668cc9-5qcpz_94e58ef4-4b8d-438b-91a3-f4b0be69234a/reload/0.log"
Sep 18 07:09:47.234727 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:47.234701    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-886668cc9-5qcpz_94e58ef4-4b8d-438b-91a3-f4b0be69234a/kube-rbac-proxy/0.log"
Sep 18 07:09:49.693147 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.693111    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4"]
Sep 18 07:09:49.693556 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.693541    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14e54543-00a6-4a77-87a3-21be77cf9925" containerName="maas-api"
Sep 18 07:09:49.693556 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.693555    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="14e54543-00a6-4a77-87a3-21be77cf9925" containerName="maas-api"
Sep 18 07:09:49.693631 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.693627    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="14e54543-00a6-4a77-87a3-21be77cf9925" containerName="maas-api"
Sep 18 07:09:49.696666 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.696651    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4"
Sep 18 07:09:49.703964 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.703939    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-87c5n\"/\"default-dockercfg-h95g6\""
Sep 18 07:09:49.704905 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.704887    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-87c5n\"/\"openshift-service-ca.crt\""
Sep 18 07:09:49.708201 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.708188    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-87c5n\"/\"kube-root-ca.crt\""
Sep 18 07:09:49.716106 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.716083    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4"]
Sep 18 07:09:49.868555 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.868521    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2639519f-adf3-4e69-bc16-68c80d26c8c0-sys\") pod \"perf-node-gather-daemonset-hvzx4\" (UID: \"2639519f-adf3-4e69-bc16-68c80d26c8c0\") " pod="openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4"
Sep 18 07:09:49.868555 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.868558    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc6fw\" (UniqueName: \"kubernetes.io/projected/2639519f-adf3-4e69-bc16-68c80d26c8c0-kube-api-access-mc6fw\") pod \"perf-node-gather-daemonset-hvzx4\" (UID: \"2639519f-adf3-4e69-bc16-68c80d26c8c0\") " pod="openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4"
Sep 18 07:09:49.868790 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.868595    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2639519f-adf3-4e69-bc16-68c80d26c8c0-lib-modules\") pod \"perf-node-gather-daemonset-hvzx4\" (UID: \"2639519f-adf3-4e69-bc16-68c80d26c8c0\") " pod="openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4"
Sep 18 07:09:49.868790 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.868649    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2639519f-adf3-4e69-bc16-68c80d26c8c0-proc\") pod \"perf-node-gather-daemonset-hvzx4\" (UID: \"2639519f-adf3-4e69-bc16-68c80d26c8c0\") " pod="openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4"
Sep 18 07:09:49.868790 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.868668    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2639519f-adf3-4e69-bc16-68c80d26c8c0-podres\") pod \"perf-node-gather-daemonset-hvzx4\" (UID: \"2639519f-adf3-4e69-bc16-68c80d26c8c0\") " pod="openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4"
Sep 18 07:09:49.969900 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.969820    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2639519f-adf3-4e69-bc16-68c80d26c8c0-sys\") pod \"perf-node-gather-daemonset-hvzx4\" (UID: \"2639519f-adf3-4e69-bc16-68c80d26c8c0\") " pod="openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4"
Sep 18 07:09:49.969900 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.969853    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc6fw\" (UniqueName: \"kubernetes.io/projected/2639519f-adf3-4e69-bc16-68c80d26c8c0-kube-api-access-mc6fw\") pod \"perf-node-gather-daemonset-hvzx4\" (UID: \"2639519f-adf3-4e69-bc16-68c80d26c8c0\") " pod="openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4"
Sep 18 07:09:49.969900 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.969886    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2639519f-adf3-4e69-bc16-68c80d26c8c0-lib-modules\") pod \"perf-node-gather-daemonset-hvzx4\" (UID: \"2639519f-adf3-4e69-bc16-68c80d26c8c0\") " pod="openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4"
Sep 18 07:09:49.970179 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.969916    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2639519f-adf3-4e69-bc16-68c80d26c8c0-proc\") pod \"perf-node-gather-daemonset-hvzx4\" (UID: \"2639519f-adf3-4e69-bc16-68c80d26c8c0\") " pod="openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4"
Sep 18 07:09:49.970179 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.969942    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2639519f-adf3-4e69-bc16-68c80d26c8c0-podres\") pod \"perf-node-gather-daemonset-hvzx4\" (UID: \"2639519f-adf3-4e69-bc16-68c80d26c8c0\") " pod="openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4"
Sep 18 07:09:49.970179 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.969946    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2639519f-adf3-4e69-bc16-68c80d26c8c0-sys\") pod \"perf-node-gather-daemonset-hvzx4\" (UID: \"2639519f-adf3-4e69-bc16-68c80d26c8c0\") " pod="openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4"
Sep 18 07:09:49.970179 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.970040    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2639519f-adf3-4e69-bc16-68c80d26c8c0-podres\") pod \"perf-node-gather-daemonset-hvzx4\" (UID: \"2639519f-adf3-4e69-bc16-68c80d26c8c0\") " pod="openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4"
Sep 18 07:09:49.970179 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.970049    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2639519f-adf3-4e69-bc16-68c80d26c8c0-proc\") pod \"perf-node-gather-daemonset-hvzx4\" (UID: \"2639519f-adf3-4e69-bc16-68c80d26c8c0\") " pod="openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4"
Sep 18 07:09:49.970179 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.970040    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2639519f-adf3-4e69-bc16-68c80d26c8c0-lib-modules\") pod \"perf-node-gather-daemonset-hvzx4\" (UID: \"2639519f-adf3-4e69-bc16-68c80d26c8c0\") " pod="openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4"
Sep 18 07:09:49.979389 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:49.979355    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc6fw\" (UniqueName: \"kubernetes.io/projected/2639519f-adf3-4e69-bc16-68c80d26c8c0-kube-api-access-mc6fw\") pod \"perf-node-gather-daemonset-hvzx4\" (UID: \"2639519f-adf3-4e69-bc16-68c80d26c8c0\") " pod="openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4"
Sep 18 07:09:50.006385 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:50.006359    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4"
Sep 18 07:09:50.282119 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:50.282034    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4"]
Sep 18 07:09:50.285332 ip-10-0-128-30 kubenswrapper[2568]: W0918 07:09:50.285306    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2639519f_adf3_4e69_bc16_68c80d26c8c0.slice/crio-77f17f04d06a2b0bb470e5be8940118b071a50d25fb973941fb4f1898e177d55 WatchSource:0}: Error finding container 77f17f04d06a2b0bb470e5be8940118b071a50d25fb973941fb4f1898e177d55: Status 404 returned error can't find the container with id 77f17f04d06a2b0bb470e5be8940118b071a50d25fb973941fb4f1898e177d55
Sep 18 07:09:50.406052 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:50.406026    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-797df4568f-g8jcs_042d2099-832e-4b63-9da0-88f08a90e8ce/console/0.log"
Sep 18 07:09:50.455887 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:50.455856    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6594f4c75-8l8dt_05b3941b-0717-4241-9cf7-a45764dcb8db/download-server/0.log"
Sep 18 07:09:50.988949 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:50.988912    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4" event={"ID":"2639519f-adf3-4e69-bc16-68c80d26c8c0","Type":"ContainerStarted","Data":"e8b6dbc5c58e78a803b60ffd967c56b553b17462cf0f1bfa70a64e4182c96d20"}
Sep 18 07:09:50.989357 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:50.988970    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4" event={"ID":"2639519f-adf3-4e69-bc16-68c80d26c8c0","Type":"ContainerStarted","Data":"77f17f04d06a2b0bb470e5be8940118b071a50d25fb973941fb4f1898e177d55"}
Sep 18 07:09:50.989357 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:50.989010    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4"
Sep 18 07:09:51.016038 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:51.015982    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4" podStartSLOduration=2.01596803 podStartE2EDuration="2.01596803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 07:09:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 07:09:51.014331397 +0000 UTC m=+2856.695859141" watchObservedRunningTime="2026-09-18 07:09:51.01596803 +0000 UTC m=+2856.697495773"
Sep 18 07:09:52.707075 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:52.707042    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jdjzn_4a5817a8-34ca-4d89-8543-76631f2d2d62/dns/0.log"
Sep 18 07:09:52.744618 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:52.744590    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jdjzn_4a5817a8-34ca-4d89-8543-76631f2d2d62/kube-rbac-proxy/0.log"
Sep 18 07:09:52.952129 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:52.952101    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-nrnz8_ead9d685-da05-4132-b36c-4887546472f2/dns-node-resolver/0.log"
Sep 18 07:09:53.650168 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:53.650136    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-db56994c6-v26pk_3b8ab2d6-1229-40ec-ae1e-4e2d2f7f66b2/registry/0.log"
Sep 18 07:09:53.696291 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:53.696263    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6lmr2_612baaf8-db1e-44ca-993e-836959f23b2d/node-ca/0.log"
Sep 18 07:09:55.332990 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:55.332955    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bbd68dc59-6grwd_415ce0db-109b-4944-b4aa-2ed7733a8b0e/kube-auth-proxy/0.log"
Sep 18 07:09:55.401748 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:55.401724    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-f2pj4_0484017e-ee10-41d4-b810-b3a80f6382da/istio-proxy/0.log"
Sep 18 07:09:55.404852 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:55.404831    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-f2pj4_0484017e-ee10-41d4-b810-b3a80f6382da/istio-proxy/1.log"
Sep 18 07:09:56.580748 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:56.580719    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wz2zs_d1f10a14-3b4a-4232-a2b0-5328c2384cb8/serve-healthcheck-canary/0.log"
Sep 18 07:09:57.003960 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:57.003934    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-87c5n/perf-node-gather-daemonset-hvzx4"
Sep 18 07:09:57.313894 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:57.313812    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rqpls_9f412820-0d80-43dc-94bd-fd4b5a773fde/kube-rbac-proxy/0.log"
Sep 18 07:09:57.337140 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:57.337115    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rqpls_9f412820-0d80-43dc-94bd-fd4b5a773fde/exporter/0.log"
Sep 18 07:09:57.359775 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:57.359756    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rqpls_9f412820-0d80-43dc-94bd-fd4b5a773fde/extractor/0.log"
Sep 18 07:09:59.824777 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:09:59.824747    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-jz5zd_d377a298-99f3-4042-98c8-303150a24782/manager/0.log"
Sep 18 07:10:00.175256 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:10:00.175210    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-5dl2q_24e331ac-1ca6-484e-a96a-2603f3f51732/manager/0.log"
Sep 18 07:10:00.281519 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:10:00.281487    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bbd68dc59-6grwd_415ce0db-109b-4944-b4aa-2ed7733a8b0e/kube-auth-proxy/0.log"
Sep 18 07:10:00.384628 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:10:00.384537    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-f2pj4_0484017e-ee10-41d4-b810-b3a80f6382da/istio-proxy/0.log"
Sep 18 07:10:00.386895 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:10:00.386876    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-f2pj4_0484017e-ee10-41d4-b810-b3a80f6382da/istio-proxy/1.log"
Sep 18 07:10:00.406071 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:10:00.406044    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-nfc2b_8947c3c0-f29d-44a8-a085-6877f589af98/server/0.log"
Sep 18 07:10:00.875039 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:10:00.875007    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fd575c64f-xv5qk_54caad4e-0c6a-43f8-bdc3-eabffaaf7c19/manager/0.log"
Sep 18 07:10:00.927677 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:10:00.927652    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6fd575c64f-xv5qk_54caad4e-0c6a-43f8-bdc3-eabffaaf7c19/copy-manifests/0.log"
Sep 18 07:10:02.711330 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:10:02.711304    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5bbd68dc59-6grwd_415ce0db-109b-4944-b4aa-2ed7733a8b0e/kube-auth-proxy/0.log"
Sep 18 07:10:02.766323 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:10:02.766284    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-f2pj4_0484017e-ee10-41d4-b810-b3a80f6382da/istio-proxy/0.log"
Sep 18 07:10:02.770516 ip-10-0-128-30 kubenswrapper[2568]: I0918 07:10:02.770487    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-f2pj4_0484017e-ee10-41d4-b810-b3a80f6382da/istio-proxy/1.log"