Sep 14 07:48:17.439844 ip-10-0-138-28 systemd[1]: Starting Kubernetes Kubelet...
Sep 14 07:48:17.778160 ip-10-0-138-28 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 07:48:17.778160 ip-10-0-138-28 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 14 07:48:17.778160 ip-10-0-138-28 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 07:48:17.778160 ip-10-0-138-28 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 14 07:48:17.778160 ip-10-0-138-28 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 07:48:17.779692 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.779599    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 14 07:48:17.784727 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784705    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:48:17.784727 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784722    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:48:17.784727 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784728    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:48:17.784727 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784731    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:48:17.784727 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784735    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:48:17.785019 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784740    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:48:17.785019 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784746    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:48:17.785019 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784752    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:48:17.785019 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784757    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:48:17.785019 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784761    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:48:17.785019 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784765    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:48:17.785019 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784769    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:48:17.785019 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784774    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:48:17.785019 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784779    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:48:17.785019 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784783    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:48:17.785019 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784786    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:48:17.785019 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784790    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:48:17.785019 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784795    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:48:17.785019 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784799    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:48:17.785019 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784803    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:48:17.785019 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784806    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:48:17.785019 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784810    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:48:17.785019 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784814    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:48:17.785019 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784818    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:48:17.785848 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784822    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:48:17.785848 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784826    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:48:17.785848 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784830    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:48:17.785848 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784834    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:48:17.785848 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784838    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:48:17.785848 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784842    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:48:17.785848 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784846    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:48:17.785848 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784850    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:48:17.785848 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784854    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:48:17.785848 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784859    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:48:17.785848 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784863    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:48:17.785848 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784867    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:48:17.785848 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784872    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:48:17.785848 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784879    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:48:17.785848 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784883    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:48:17.785848 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784888    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:48:17.785848 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784895    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:48:17.785848 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784900    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:48:17.785848 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784905    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:48:17.786677 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784909    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:48:17.786677 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784914    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:48:17.786677 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784918    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:48:17.786677 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784922    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:48:17.786677 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784926    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:48:17.786677 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784931    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:48:17.786677 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784935    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:48:17.786677 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784939    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:48:17.786677 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784943    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:48:17.786677 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784947    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:48:17.786677 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784951    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:48:17.786677 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784956    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:48:17.786677 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784960    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:48:17.786677 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784964    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:48:17.786677 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784968    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:48:17.786677 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784972    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:48:17.786677 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784976    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:48:17.786677 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784980    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:48:17.786677 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784985    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:48:17.786677 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784990    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:48:17.787526 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784994    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:48:17.787526 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.784998    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:48:17.787526 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785002    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:48:17.787526 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785006    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:48:17.787526 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785010    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:48:17.787526 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785014    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:48:17.787526 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785020    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:48:17.787526 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785026    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:48:17.787526 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785030    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:48:17.787526 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785034    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:48:17.787526 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785038    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:48:17.787526 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785043    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:48:17.787526 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785048    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:48:17.787526 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785052    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:48:17.787526 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785058    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:48:17.787526 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785064    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:48:17.787526 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785069    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:48:17.787526 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785073    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:48:17.787526 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785078    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:48:17.787526 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785082    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:48:17.788384 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785087    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:48:17.788384 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785091    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:48:17.788384 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785095    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:48:17.788384 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785100    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:48:17.788384 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785104    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:48:17.788384 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785109    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:48:17.788384 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785113    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:48:17.788384 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785117    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:48:17.788384 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785123    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:48:17.788384 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785128    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:48:17.788384 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785133    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:48:17.788384 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785137    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:48:17.788384 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785141    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:48:17.788384 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785146    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:48:17.788384 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785150    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:48:17.788384 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.785155    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:48:17.788384 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786327    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:48:17.788384 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786337    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:48:17.788384 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786342    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:48:17.788384 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786346    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:48:17.789240 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786352    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:48:17.789240 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786356    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:48:17.789240 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786361    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:48:17.789240 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786365    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:48:17.789240 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786369    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:48:17.789240 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786374    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:48:17.789240 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786379    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:48:17.789240 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786383    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:48:17.789240 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786387    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:48:17.789240 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786391    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:48:17.789240 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786396    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:48:17.789240 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786400    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:48:17.789240 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786404    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:48:17.789240 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786409    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:48:17.789240 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786413    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:48:17.789240 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786417    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:48:17.789240 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786422    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:48:17.789240 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786426    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:48:17.789240 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786430    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:48:17.789240 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786434    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:48:17.789951 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786438    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:48:17.789951 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786443    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:48:17.789951 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786448    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:48:17.789951 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786453    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:48:17.789951 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786459    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:48:17.789951 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786465    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:48:17.789951 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786470    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:48:17.789951 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786474    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:48:17.789951 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786478    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:48:17.789951 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786483    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:48:17.789951 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786487    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:48:17.789951 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786492    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:48:17.789951 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786518    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:48:17.789951 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786526    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:48:17.789951 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786531    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:48:17.789951 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786536    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:48:17.789951 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786541    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:48:17.789951 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786546    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:48:17.789951 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786551    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:48:17.790490 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786555    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:48:17.790490 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786560    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:48:17.790490 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786565    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:48:17.790490 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786570    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:48:17.790490 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786574    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:48:17.790490 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786579    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:48:17.790490 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786584    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:48:17.790490 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786589    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:48:17.790490 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786593    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:48:17.790490 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786598    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:48:17.790490 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786602    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:48:17.790490 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786607    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:48:17.790490 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786611    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:48:17.790490 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786615    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:48:17.790490 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786619    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:48:17.790490 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786624    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:48:17.790490 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786629    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:48:17.790490 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786633    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:48:17.790490 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786637    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:48:17.791018 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786643    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:48:17.791018 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786649    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:48:17.791018 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786654    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:48:17.791018 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786659    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:48:17.791018 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786663    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:48:17.791018 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786668    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:48:17.791018 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786672    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:48:17.791018 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786676    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:48:17.791018 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786681    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:48:17.791018 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786685    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:48:17.791018 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786689    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:48:17.791018 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786693    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:48:17.791018 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786697    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:48:17.791018 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786701    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:48:17.791018 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786705    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:48:17.791018 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786710    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:48:17.791018 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786714    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:48:17.791018 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786718    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:48:17.791018 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786722    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:48:17.791550 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786726    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:48:17.791550 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786731    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:48:17.791550 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786735    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:48:17.791550 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786739    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:48:17.791550 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786750    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:48:17.791550 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786755    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:48:17.791550 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786761    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:48:17.791550 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786765    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:48:17.791550 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786769    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:48:17.791550 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786774    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:48:17.791550 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786778    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:48:17.791550 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786782    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:48:17.791550 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786786    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:48:17.791550 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786791    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:48:17.791550 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786796    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:48:17.791550 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786801    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:48:17.791550 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786805    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:48:17.791550 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.786810    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:48:17.791550 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787398    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:48:17.791550 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787405    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:48:17.792294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787410    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:48:17.792294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787415    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:48:17.792294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787420    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:48:17.792294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787424    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:48:17.792294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787429    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:48:17.792294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787433    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:48:17.792294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787437    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:48:17.792294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787442    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:48:17.792294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787446    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:48:17.792294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787450    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:48:17.792294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787454    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:48:17.792294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787459    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:48:17.792294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787462    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:48:17.792294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787467    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:48:17.792294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787471    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:48:17.792294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787475    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:48:17.792294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787479    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:48:17.792294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787484    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:48:17.792294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787488    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:48:17.792294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787492    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:48:17.792811 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787581    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:48:17.792811 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787588    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:48:17.792811 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787593    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:48:17.792811 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787598    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:48:17.792811 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787602    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:48:17.792811 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787606    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:48:17.792811 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787611    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:48:17.792811 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787616    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:48:17.792811 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787620    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:48:17.792811 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787624    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:48:17.792811 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787628    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:48:17.792811 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787632    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:48:17.792811 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787636    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:48:17.792811 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787640    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:48:17.792811 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787645    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:48:17.792811 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787650    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:48:17.792811 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787654    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:48:17.792811 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787658    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:48:17.792811 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787662    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:48:17.792811 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787667    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:48:17.793405 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787671    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:48:17.793405 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787677    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:48:17.793405 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787681    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:48:17.793405 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787685    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:48:17.793405 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787690    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:48:17.793405 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787695    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:48:17.793405 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787699    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:48:17.793405 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787704    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:48:17.793405 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787708    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:48:17.793405 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787712    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:48:17.793405 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787716    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:48:17.793405 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787720    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:48:17.793405 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787725    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:48:17.793405 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787730    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:48:17.793405 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787734    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:48:17.793405 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787749    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:48:17.793405 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787753    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:48:17.793405 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787757    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:48:17.793405 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787761    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:48:17.793890 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787769    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:48:17.793890 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787775    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:48:17.793890 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787780    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:48:17.793890 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787785    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:48:17.793890 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787789    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:48:17.793890 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787794    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:48:17.793890 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787798    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:48:17.793890 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787802    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:48:17.793890 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787807    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:48:17.793890 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787812    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:48:17.793890 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787816    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:48:17.793890 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787823    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:48:17.793890 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787828    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:48:17.793890 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787832    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:48:17.793890 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787837    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:48:17.793890 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787841    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:48:17.793890 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787845    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:48:17.793890 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787849    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:48:17.793890 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787857    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:48:17.794378 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787864    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:48:17.794378 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787869    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:48:17.794378 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787874    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:48:17.794378 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787879    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:48:17.794378 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787883    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:48:17.794378 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787888    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:48:17.794378 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787892    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:48:17.794378 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787896    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:48:17.794378 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787900    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:48:17.794378 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787905    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:48:17.794378 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787909    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:48:17.794378 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787914    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:48:17.794378 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787918    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:48:17.794378 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787922    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:48:17.794378 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787927    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:48:17.794378 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787930    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:48:17.794378 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787935    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:48:17.794378 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787939    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:48:17.794378 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.787943    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:48:17.794378 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788046    2562 flags.go:64] FLAG: --address="0.0.0.0"
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788097    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788108    2562 flags.go:64] FLAG: --anonymous-auth="true"
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788116    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788123    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788128    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788134    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788141    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788146    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788151    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788157    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788163    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788168    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788173    2562 flags.go:64] FLAG: --cgroup-root=""
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788178    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788183    2562 flags.go:64] FLAG: --client-ca-file=""
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788188    2562 flags.go:64] FLAG: --cloud-provider="external"
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788193    2562 flags.go:64] FLAG: --cluster-dns="[]"
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788199    2562 flags.go:64] FLAG: --cluster-domain=""
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788204    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788209    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788214    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788219    2562 flags.go:64] FLAG: --container-log-max-files="5"
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788225    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 14 07:48:17.794901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788230    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788236    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788242    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788246    2562 flags.go:64] FLAG: --contention-profiling="false"
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788251    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788256    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788261    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788266    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788273    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788278    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788283    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788287    2562 flags.go:64] FLAG: --enable-load-reader="false"
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788293    2562 flags.go:64] FLAG: --enable-server="true"
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788298    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788306    2562 flags.go:64] FLAG: --event-burst="100"
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788311    2562 flags.go:64] FLAG: --event-qps="50"
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788316    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788321    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788325    2562 flags.go:64] FLAG: --eviction-hard=""
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788335    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788340    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788345    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788350    2562 flags.go:64] FLAG: --eviction-soft=""
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788355    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788360    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 14 07:48:17.795487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788365    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788370    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788374    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788379    2562 flags.go:64] FLAG: --fail-swap-on="true"
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788383    2562 flags.go:64] FLAG: --feature-gates=""
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788389    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788394    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788398    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788405    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788410    2562 flags.go:64] FLAG: --healthz-port="10248"
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788415    2562 flags.go:64] FLAG: --help="false"
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788420    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-138-28.ec2.internal"
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788425    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788430    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788434    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788440    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788446    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788450    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788455    2562 flags.go:64] FLAG: --image-service-endpoint=""
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788460    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788466    2562 flags.go:64] FLAG: --kube-api-burst="100"
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788471    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788476    2562 flags.go:64] FLAG: --kube-api-qps="50"
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788481    2562 flags.go:64] FLAG: --kube-reserved=""
Sep 14 07:48:17.796382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788486    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788490    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788516    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788522    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788527    2562 flags.go:64] FLAG: --lock-file=""
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788531    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788537    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788542    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788550    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788554    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788559    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788564    2562 flags.go:64] FLAG: --logging-format="text"
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788569    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788574    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788579    2562 flags.go:64] FLAG: --manifest-url=""
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788584    2562 flags.go:64] FLAG: --manifest-url-header=""
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788594    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788600    2562 flags.go:64] FLAG: --max-open-files="1000000"
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788608    2562 flags.go:64] FLAG: --max-pods="110"
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788613    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788618    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788622    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788627    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788632    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788637    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 14 07:48:17.797151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788641    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788653    2562 flags.go:64] FLAG: --node-status-max-images="50"
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788658    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788663    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788669    2562 flags.go:64] FLAG: --pod-cidr=""
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788673    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788681    2562 flags.go:64] FLAG: --pod-manifest-path=""
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788686    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788692    2562 flags.go:64] FLAG: --pods-per-core="0"
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788696    2562 flags.go:64] FLAG: --port="10250"
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788703    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788708    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-015d85957db4e50d0"
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788713    2562 flags.go:64] FLAG: --qos-reserved=""
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788718    2562 flags.go:64] FLAG: --read-only-port="10255"
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788723    2562 flags.go:64] FLAG: --register-node="true"
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788727    2562 flags.go:64] FLAG: --register-with-taints=""
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788733    2562 flags.go:64] FLAG: --registry-burst="10"
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788738    2562 flags.go:64] FLAG: --registry-qps="5"
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788743    2562 flags.go:64] FLAG: --reserved-cpus=""
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788748    2562 flags.go:64] FLAG: --reserved-memory=""
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788758    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788762    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788767    2562 flags.go:64] FLAG: --rotate-certificates="false"
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788784    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788789    2562 flags.go:64] FLAG: --runonce="false"
Sep 14 07:48:17.797795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788794    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788799    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788804    2562 flags.go:64] FLAG: --seccomp-default="false"
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788809    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788814    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788819    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788824    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788829    2562 flags.go:64] FLAG: --storage-driver-password="root"
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788834    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788838    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788843    2562 flags.go:64] FLAG: --storage-driver-user="root"
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788848    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788854    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788859    2562 flags.go:64] FLAG: --system-cgroups=""
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788864    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788873    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788877    2562 flags.go:64] FLAG: --tls-cert-file=""
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788884    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788890    2562 flags.go:64] FLAG: --tls-min-version=""
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788896    2562 flags.go:64] FLAG: --tls-private-key-file=""
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788901    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788905    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788910    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788915    2562 flags.go:64] FLAG: --v="2"
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788921    2562 flags.go:64] FLAG: --version="false"
Sep 14 07:48:17.798411 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788928    2562 flags.go:64] FLAG: --vmodule=""
Sep 14 07:48:17.799056 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788934    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 14 07:48:17.799056 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.788940    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 14 07:48:17.799056 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789121    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:48:17.799056 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789128    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:48:17.799056 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789134    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:48:17.799056 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789139    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:48:17.799056 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789143    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:48:17.799056 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789147    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:48:17.799056 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789151    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:48:17.799056 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789155    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:48:17.799056 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789159    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:48:17.799056 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789164    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:48:17.799056 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789168    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:48:17.799056 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789172    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:48:17.799056 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789176    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:48:17.799056 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789180    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:48:17.799056 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789184    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:48:17.799056 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789188    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:48:17.799056 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789192    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:48:17.799056 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789197    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:48:17.799602 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789202    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:48:17.799602 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789207    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:48:17.799602 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789211    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:48:17.799602 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789217    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:48:17.799602 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789221    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:48:17.799602 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789226    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:48:17.799602 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789230    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:48:17.799602 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789234    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:48:17.799602 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789238    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:48:17.799602 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789241    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:48:17.799602 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789245    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:48:17.799602 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789250    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:48:17.799602 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789253    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:48:17.799602 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789258    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:48:17.799602 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789261    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:48:17.799602 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789265    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:48:17.799602 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789271    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:48:17.799602 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789276    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:48:17.799602 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789280    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:48:17.799602 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789285    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:48:17.800092 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789291    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:48:17.800092 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789297    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:48:17.800092 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789303    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:48:17.800092 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789307    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:48:17.800092 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789311    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:48:17.800092 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789316    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:48:17.800092 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789320    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:48:17.800092 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789324    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:48:17.800092 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789329    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:48:17.800092 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789333    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:48:17.800092 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789337    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:48:17.800092 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789341    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:48:17.800092 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789346    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:48:17.800092 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789350    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:48:17.800092 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789354    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:48:17.800092 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789360    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:48:17.800092 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789364    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:48:17.800092 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789368    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:48:17.800092 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789372    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:48:17.800574 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789378    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:48:17.800574 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789383    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:48:17.800574 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789387    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:48:17.800574 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789391    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:48:17.800574 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789396    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:48:17.800574 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789400    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:48:17.800574 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789404    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:48:17.800574 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789408    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:48:17.800574 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789413    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:48:17.800574 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789419    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:48:17.800574 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789423    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:48:17.800574 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789427    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:48:17.800574 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789431    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:48:17.800574 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789435    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:48:17.800574 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789440    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:48:17.800574 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789444    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:48:17.800574 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789448    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:48:17.800574 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789452    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:48:17.800574 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789457    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:48:17.801046 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789461    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:48:17.801046 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789465    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:48:17.801046 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789469    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:48:17.801046 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789473    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:48:17.801046 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789477    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:48:17.801046 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789482    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:48:17.801046 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789486    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:48:17.801046 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789491    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:48:17.801046 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789495    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:48:17.801046 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789517    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:48:17.801046 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789522    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:48:17.801046 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789526    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:48:17.801046 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789530    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:48:17.801046 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789534    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:48:17.801046 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789538    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:48:17.801046 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789541    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:48:17.801046 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789545    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:48:17.801046 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789552    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:48:17.801046 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789558    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:48:17.801046 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789563    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:48:17.801563 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789567    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:48:17.801563 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789572    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:48:17.801563 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.789578    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:48:17.801563 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.790197    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 07:48:17.801563 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.799235    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 14 07:48:17.801563 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.799251    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 14 07:48:17.801563 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.799901    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 14 07:48:17.801563 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.799908    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 14 07:48:17.801563 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.799965    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:48:17.801563 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.799970    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:48:17.801563 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.799973    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:48:17.801563 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.799976    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:48:17.801563 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.799979    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:48:17.801563 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.799982    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:48:17.801563 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.799985    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:48:17.801947 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.799990    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:48:17.801947 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.799994    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:48:17.801947 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.799997    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:48:17.801947 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.799999    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:48:17.801947 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800002    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:48:17.801947 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800006    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:48:17.801947 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800010    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:48:17.801947 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800014    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:48:17.801947 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800017    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:48:17.801947 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800020    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:48:17.801947 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800023    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:48:17.801947 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800025    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:48:17.801947 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800028    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:48:17.801947 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800031    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:48:17.801947 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800033    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:48:17.801947 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800036    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:48:17.801947 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800038    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:48:17.801947 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800041    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:48:17.801947 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800043    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:48:17.802413 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800046    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:48:17.802413 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800049    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:48:17.802413 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800052    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:48:17.802413 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800055    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:48:17.802413 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800058    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:48:17.802413 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800061    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:48:17.802413 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800063    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:48:17.802413 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800066    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:48:17.802413 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800069    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:48:17.802413 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800071    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:48:17.802413 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800074    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:48:17.802413 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800076    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:48:17.802413 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800079    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:48:17.802413 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800081    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:48:17.802413 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800084    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:48:17.802413 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800087    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:48:17.802413 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800089    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:48:17.802413 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800092    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:48:17.802413 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800095    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:48:17.802413 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800098    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:48:17.802936 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800102    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:48:17.802936 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800105    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:48:17.802936 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800107    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:48:17.802936 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800110    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:48:17.802936 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800113    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:48:17.802936 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800116    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:48:17.802936 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800119    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:48:17.802936 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800121    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:48:17.802936 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800124    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:48:17.802936 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800126    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:48:17.802936 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800130    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:48:17.802936 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800133    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:48:17.802936 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800136    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:48:17.802936 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800139    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:48:17.802936 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800142    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:48:17.802936 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800146    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:48:17.802936 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800149    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:48:17.802936 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800152    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:48:17.802936 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800155    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:48:17.803444 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800158    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:48:17.803444 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800161    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:48:17.803444 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800163    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:48:17.803444 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800166    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:48:17.803444 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800168    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:48:17.803444 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800171    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:48:17.803444 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800174    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:48:17.803444 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800176    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:48:17.803444 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800179    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:48:17.803444 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800181    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:48:17.803444 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800183    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:48:17.803444 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800186    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:48:17.803444 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800189    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:48:17.803444 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800191    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:48:17.803444 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800195    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:48:17.803444 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800198    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:48:17.803444 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800200    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:48:17.803444 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800203    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:48:17.803444 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800206    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:48:17.803444 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800208    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:48:17.803950 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800211    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:48:17.803950 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800214    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:48:17.803950 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800216    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:48:17.803950 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800219    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:48:17.803950 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800221    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:48:17.803950 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800223    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:48:17.803950 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800226    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:48:17.803950 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800229    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:48:17.803950 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800232    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:48:17.803950 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800234    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:48:17.803950 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800237    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:48:17.803950 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800240    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:48:17.803950 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800243    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:48:17.803950 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800246    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:48:17.804294 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.800250    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 07:48:17.804294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800361    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:48:17.804294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800367    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:48:17.804294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800370    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:48:17.804294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800373    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:48:17.804294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800376    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:48:17.804294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800378    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:48:17.804294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800381    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:48:17.804294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800383    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:48:17.804294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800386    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:48:17.804294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800388    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:48:17.804294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800391    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:48:17.804294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800394    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:48:17.804294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800398    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:48:17.804294 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800402    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:48:17.804744 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800405    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:48:17.804744 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800408    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:48:17.804744 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800411    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:48:17.804744 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800414    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:48:17.804744 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800417    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:48:17.804744 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800420    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:48:17.804744 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800423    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:48:17.804744 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800426    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:48:17.804744 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800428    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:48:17.804744 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800432    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:48:17.804744 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800434    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:48:17.804744 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800437    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:48:17.804744 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800440    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:48:17.804744 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800443    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:48:17.804744 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800447    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:48:17.804744 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800451    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:48:17.804744 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800454    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:48:17.804744 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800457    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:48:17.804744 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800460    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:48:17.805196 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800464    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:48:17.805196 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800466    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:48:17.805196 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800469    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:48:17.805196 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800472    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:48:17.805196 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800474    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:48:17.805196 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800477    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:48:17.805196 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800479    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:48:17.805196 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800482    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:48:17.805196 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800484    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:48:17.805196 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800487    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:48:17.805196 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800489    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:48:17.805196 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800492    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:48:17.805196 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800495    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:48:17.805196 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800508    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:48:17.805196 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800511    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:48:17.805196 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800513    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:48:17.805196 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800516    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:48:17.805196 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800519    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:48:17.805196 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800522    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:48:17.805196 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800524    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:48:17.805702 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800527    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:48:17.805702 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800530    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:48:17.805702 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800532    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:48:17.805702 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800535    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:48:17.805702 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800538    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:48:17.805702 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800541    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:48:17.805702 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800543    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:48:17.805702 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800546    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:48:17.805702 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800549    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:48:17.805702 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800551    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:48:17.805702 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800554    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:48:17.805702 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800557    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:48:17.805702 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800559    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:48:17.805702 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800562    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:48:17.805702 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800564    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:48:17.805702 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800567    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:48:17.805702 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800569    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:48:17.805702 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800572    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:48:17.805702 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800575    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:48:17.805702 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800578    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:48:17.806188 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800581    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:48:17.806188 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800584    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:48:17.806188 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800586    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:48:17.806188 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800589    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:48:17.806188 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800592    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:48:17.806188 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800595    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:48:17.806188 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800598    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:48:17.806188 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800600    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:48:17.806188 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800603    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:48:17.806188 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800605    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:48:17.806188 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800608    2562 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:48:17.806188 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800610    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:48:17.806188 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800613    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:48:17.806188 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800615    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:48:17.806188 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800618    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:48:17.806188 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800621    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:48:17.806188 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800623    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:48:17.806188 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800626    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:48:17.806188 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800629    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:48:17.806188 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800631    2562 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:48:17.806700 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800634    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:48:17.806700 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800637    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:48:17.806700 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800639    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:48:17.806700 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800642    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:48:17.806700 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800644    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:48:17.806700 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:17.800647    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:48:17.806700 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.800652    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 07:48:17.806700 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.800769    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 14 07:48:17.806700 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.803041    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 14 07:48:17.806700 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.803978    2562 server.go:1019] "Starting client certificate rotation"
Sep 14 07:48:17.806700 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.804080    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 07:48:17.806700 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.804112    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 07:48:17.806700 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.806242    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 07:48:17.810160 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.810146    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 07:48:17.820037 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.820017    2562 log.go:25] "Validated CRI v1 runtime API"
Sep 14 07:48:17.824478 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.824461    2562 log.go:25] "Validated CRI v1 image API"
Sep 14 07:48:17.827557 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.827538    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 14 07:48:17.830232 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.830211    2562 fs.go:135] Filesystem UUIDs: map[15491eee-0fc1-4096-91e4-5a6406e894a7:/dev/nvme0n1p4 4ef78069-fac2-4ef9-b23a-ad6c0739beff:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Sep 14 07:48:17.830295 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.830230    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 14 07:48:17.835672 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.835561    2562 manager.go:216] Machine: {Timestamp:2026-09-14 07:48:17.834428953 +0000 UTC m=+0.290878938 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:2983703 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec253a7933d4495114b36585b74a4b56 SystemUUID:ec253a79-33d4-4951-14b3-6585b74a4b56 BootID:c4f17906-8349-4b38-b4cb-1a9b0fe8de26 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e7:e8:c8:d1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e7:e8:c8:d1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:0e:eb:e8:64:3a:45 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 14 07:48:17.835672 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.835666    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 14 07:48:17.835782 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.835746    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 14 07:48:17.838538 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.838512    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 14 07:48:17.838675 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.838541    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-28.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 14 07:48:17.839120 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.839109    2562 topology_manager.go:138] "Creating topology manager with none policy"
Sep 14 07:48:17.839153 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.839121    2562 container_manager_linux.go:309] "Creating device plugin manager"
Sep 14 07:48:17.839153 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.839128    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 07:48:17.839708 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.839698    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 07:48:17.840726 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.840716    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 14 07:48:17.844767 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.844746    2562 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 07:48:17.844873 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.844863    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 14 07:48:17.844903 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.844893    2562 kubelet.go:486] "Attempting to sync node with API server"
Sep 14 07:48:17.844903 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.844902    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 14 07:48:17.844954 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.844914    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 14 07:48:17.844954 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.844923    2562 kubelet.go:398] "Adding apiserver pod source"
Sep 14 07:48:17.844954 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.844932    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 14 07:48:17.845341 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.845316    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 07:48:17.845341 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.845339    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 07:48:17.845476 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.845454    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 07:48:17.847165 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.847149    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 14 07:48:17.850027 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.850014    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 14 07:48:17.850086 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.850043    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 14 07:48:17.851788 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.851774    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 14 07:48:17.851836 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.851792    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 14 07:48:17.851836 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.851798    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 14 07:48:17.851836 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.851803    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 14 07:48:17.851836 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.851811    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 14 07:48:17.851836 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.851818    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 14 07:48:17.851836 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.851834    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 14 07:48:17.851993 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.851840    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 14 07:48:17.851993 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.851848    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 14 07:48:17.851993 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.851855    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 14 07:48:17.851993 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.851871    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 14 07:48:17.851993 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.851876    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 14 07:48:17.852987 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.852977    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 14 07:48:17.852987 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.852987    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 14 07:48:17.856913 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.856888    2562 server.go:1268] "Started kubelet"
Sep 14 07:48:17.857141 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.857111    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 14 07:48:17.857316 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.857098    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 14 07:48:17.857316 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.857189    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 14 07:48:17.857981 ip-10-0-138-28 systemd[1]: Started Kubernetes Kubelet.
Sep 14 07:48:17.861647 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.861618    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 14 07:48:17.861724 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.861627    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-28.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 14 07:48:17.861838 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:17.861743    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 14 07:48:17.862150 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:17.862094    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-28.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 14 07:48:17.863302 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.863285    2562 server.go:310] "Adding debug handlers to kubelet server"
Sep 14 07:48:17.864473 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:17.864426    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 14 07:48:17.865398 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:17.864563    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-28.ec2.internal.18d5207147c0ec20  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-28.ec2.internal,UID:ip-10-0-138-28.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-28.ec2.internal,},FirstTimestamp:2026-09-14 07:48:17.856859168 +0000 UTC m=+0.313309152,LastTimestamp:2026-09-14 07:48:17.856859168 +0000 UTC m=+0.313309152,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-28.ec2.internal,}"
Sep 14 07:48:17.867328 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.867312    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kfgmw"
Sep 14 07:48:17.868028 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.868002    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 14 07:48:17.868462 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.868446    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 14 07:48:17.868558 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:17.868527    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-28.ec2.internal\" not found"
Sep 14 07:48:17.869093 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.869075    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 14 07:48:17.869156 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.869077    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 14 07:48:17.869156 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.869108    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 14 07:48:17.869250 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.869196    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 14 07:48:17.869250 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.869205    2562 factory.go:55] Registering systemd factory
Sep 14 07:48:17.869250 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.869211    2562 factory.go:223] Registration of the systemd container factory successfully
Sep 14 07:48:17.869250 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.869225    2562 reconstruct.go:97] "Volume reconstruction finished"
Sep 14 07:48:17.869250 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.869237    2562 reconciler.go:29] "Reconciler: start to sync state"
Sep 14 07:48:17.869470 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.869410    2562 factory.go:153] Registering CRI-O factory
Sep 14 07:48:17.869470 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.869432    2562 factory.go:223] Registration of the crio container factory successfully
Sep 14 07:48:17.869470 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.869452    2562 factory.go:103] Registering Raw factory
Sep 14 07:48:17.869470 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.869463    2562 manager.go:1196] Started watching for new ooms in manager
Sep 14 07:48:17.869862 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.869844    2562 manager.go:349] Starting recovery of all containers
Sep 14 07:48:17.876668 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.876468    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kfgmw"
Sep 14 07:48:17.877729 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:17.877701    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 14 07:48:17.878269 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.878253    2562 manager.go:354] Recovery completed
Sep 14 07:48:17.878550 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:17.878519    2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-28.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 14 07:48:17.882991 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.882956    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:48:17.884715 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.884696    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-28.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:48:17.884798 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.884729    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-28.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:48:17.884798 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.884739    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-28.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:48:17.885799 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.885783    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 14 07:48:17.885799 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.885793    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:48:17.885799 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.885800    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 14 07:48:17.885927 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.885817    2562 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 07:48:17.886563 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.886517    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-28.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:48:17.886563 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.886543    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-28.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:48:17.886563 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.886555    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-28.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:48:17.887784 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.887771    2562 policy_none.go:49] "None policy: Start"
Sep 14 07:48:17.887842 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.887787    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 14 07:48:17.887842 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.887797    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 14 07:48:17.889691 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.889680    2562 policy_none.go:47] "Start"
Sep 14 07:48:17.944069 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.917934    2562 manager.go:337] "Starting Device Plugin manager"
Sep 14 07:48:17.944069 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:17.917971    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 14 07:48:17.944069 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.917981    2562 server.go:87] "Starting device plugin registration server"
Sep 14 07:48:17.944069 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.918352    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 14 07:48:17.944069 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.918376    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 14 07:48:17.944069 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.918526    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 14 07:48:17.944069 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.919151    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 14 07:48:17.944069 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.919162    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 14 07:48:17.944069 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:17.919266    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 14 07:48:17.944069 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:17.919306    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-28.ec2.internal\" not found"
Sep 14 07:48:17.968095 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:17.968071    2562 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2598 into the system container \"/system.slice\": <nil>"
Sep 14 07:48:18.019355 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.019324    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:48:18.020359 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.020341    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-28.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:48:18.020462 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.020374    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-28.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:48:18.020462 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.020385    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-28.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:48:18.020462 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.020413    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-28.ec2.internal"
Sep 14 07:48:18.029164 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.029092    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-28.ec2.internal"
Sep 14 07:48:18.029164 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:18.029117    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-28.ec2.internal\": node \"ip-10-0-138-28.ec2.internal\" not found"
Sep 14 07:48:18.045889 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:18.045865    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-28.ec2.internal\" not found"
Sep 14 07:48:18.104752 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.104709    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 14 07:48:18.106038 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.106007    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 14 07:48:18.106159 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.106062    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 14 07:48:18.106159 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.106088    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 14 07:48:18.106159 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:18.106132    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 14 07:48:18.108436 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.108414    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:48:18.146748 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:18.146712    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-28.ec2.internal\" not found"
Sep 14 07:48:18.207234 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.207195    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-28.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-28.ec2.internal"]
Sep 14 07:48:18.207325 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.207292    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:48:18.210260 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.210242    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-28.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:48:18.210365 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.210273    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-28.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:48:18.210365 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.210288    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-28.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:48:18.212667 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.212654    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:48:18.212802 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.212787    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-28.ec2.internal"
Sep 14 07:48:18.212856 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.212818    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:48:18.213431 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.213415    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-28.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:48:18.213431 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.213422    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-28.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:48:18.213634 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.213441    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-28.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:48:18.213634 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.213448    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-28.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:48:18.213634 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.213454    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-28.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:48:18.213634 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.213463    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-28.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:48:18.215817 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.215799    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-28.ec2.internal"
Sep 14 07:48:18.215915 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.215829    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:48:18.219804 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.219790    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-28.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:48:18.219899 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.219819    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-28.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:48:18.219899 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.219833    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-28.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:48:18.230056 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:18.230033    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-28.ec2.internal\" not found" node="ip-10-0-138-28.ec2.internal"
Sep 14 07:48:18.235676 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:18.235660    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-28.ec2.internal\" not found" node="ip-10-0-138-28.ec2.internal"
Sep 14 07:48:18.247065 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:18.247039    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-28.ec2.internal\" not found"
Sep 14 07:48:18.271764 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.271730    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb7e8b3289b4b5c21abcc1b439d2f0dd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-28.ec2.internal\" (UID: \"eb7e8b3289b4b5c21abcc1b439d2f0dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-28.ec2.internal"
Sep 14 07:48:18.271764 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.271762    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ef4817a2551205637f03cf112a16afcb-config\") pod \"kube-apiserver-proxy-ip-10-0-138-28.ec2.internal\" (UID: \"ef4817a2551205637f03cf112a16afcb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-28.ec2.internal"
Sep 14 07:48:18.271958 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.271786    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eb7e8b3289b4b5c21abcc1b439d2f0dd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-28.ec2.internal\" (UID: \"eb7e8b3289b4b5c21abcc1b439d2f0dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-28.ec2.internal"
Sep 14 07:48:18.347816 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:18.347728    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-28.ec2.internal\" not found"
Sep 14 07:48:18.372169 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.372144    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ef4817a2551205637f03cf112a16afcb-config\") pod \"kube-apiserver-proxy-ip-10-0-138-28.ec2.internal\" (UID: \"ef4817a2551205637f03cf112a16afcb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-28.ec2.internal"
Sep 14 07:48:18.372277 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.372226    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eb7e8b3289b4b5c21abcc1b439d2f0dd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-28.ec2.internal\" (UID: \"eb7e8b3289b4b5c21abcc1b439d2f0dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-28.ec2.internal"
Sep 14 07:48:18.372277 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.372249    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eb7e8b3289b4b5c21abcc1b439d2f0dd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-28.ec2.internal\" (UID: \"eb7e8b3289b4b5c21abcc1b439d2f0dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-28.ec2.internal"
Sep 14 07:48:18.372277 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.372252    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ef4817a2551205637f03cf112a16afcb-config\") pod \"kube-apiserver-proxy-ip-10-0-138-28.ec2.internal\" (UID: \"ef4817a2551205637f03cf112a16afcb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-28.ec2.internal"
Sep 14 07:48:18.372277 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.372263    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb7e8b3289b4b5c21abcc1b439d2f0dd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-28.ec2.internal\" (UID: \"eb7e8b3289b4b5c21abcc1b439d2f0dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-28.ec2.internal"
Sep 14 07:48:18.372402 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.372304    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eb7e8b3289b4b5c21abcc1b439d2f0dd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-28.ec2.internal\" (UID: \"eb7e8b3289b4b5c21abcc1b439d2f0dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-28.ec2.internal"
Sep 14 07:48:18.448350 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:18.448317    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-28.ec2.internal\" not found"
Sep 14 07:48:18.536588 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.536529    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-28.ec2.internal"
Sep 14 07:48:18.538832 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.538809    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-28.ec2.internal"
Sep 14 07:48:18.549434 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:18.549418    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-28.ec2.internal\" not found"
Sep 14 07:48:18.650601 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:18.650497    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-28.ec2.internal\" not found"
Sep 14 07:48:18.751212 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:18.751179    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-28.ec2.internal\" not found"
Sep 14 07:48:18.805359 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.805319    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 14 07:48:18.805864 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.805496    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 14 07:48:18.851394 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:18.851355    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-28.ec2.internal\" not found"
Sep 14 07:48:18.882068 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.882024    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-13 07:43:18 +0000 UTC" deadline="2028-06-02 12:08:04.180821374 +0000 UTC"
Sep 14 07:48:18.882068 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:18.882062    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15052h19m45.298763269s"
Sep 14 07:48:18.952538 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:18.952467    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-28.ec2.internal\" not found"
Sep 14 07:48:19.053294 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:19.053195    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-28.ec2.internal\" not found"
Sep 14 07:48:19.143483 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.143446    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:48:19.153659 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:19.153637    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-28.ec2.internal\" not found"
Sep 14 07:48:19.164151 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:19.164115    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef4817a2551205637f03cf112a16afcb.slice/crio-8b6eda20d5b09670fb3f9275eca4ed35dedc63a2bd111925741637dbe288bb9a WatchSource:0}: Error finding container 8b6eda20d5b09670fb3f9275eca4ed35dedc63a2bd111925741637dbe288bb9a: Status 404 returned error can't find the container with id 8b6eda20d5b09670fb3f9275eca4ed35dedc63a2bd111925741637dbe288bb9a
Sep 14 07:48:19.164909 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:19.164878    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb7e8b3289b4b5c21abcc1b439d2f0dd.slice/crio-d9ae53b099aed7fc94725aa3daa392f6e647d7e5bb4d63c4421785f427ac51cb WatchSource:0}: Error finding container d9ae53b099aed7fc94725aa3daa392f6e647d7e5bb4d63c4421785f427ac51cb: Status 404 returned error can't find the container with id d9ae53b099aed7fc94725aa3daa392f6e647d7e5bb4d63c4421785f427ac51cb
Sep 14 07:48:19.169365 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.169345    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 07:48:19.204265 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.204179    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:48:19.268758 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.268725    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-28.ec2.internal"
Sep 14 07:48:19.281471 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.281447    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 07:48:19.282485 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.282469    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-28.ec2.internal"
Sep 14 07:48:19.289607 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.289586    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 07:48:19.415514 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.415484    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:48:19.699445 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.699359    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:48:19.846880 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.846777    2562 apiserver.go:52] "Watching apiserver"
Sep 14 07:48:19.857034 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.857001    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 14 07:48:19.858066 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.857646    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-pdfcv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-28.ec2.internal","openshift-multus/multus-additional-cni-plugins-rbgj4","openshift-multus/network-metrics-daemon-4mfj7","openshift-network-operator/iptables-alerter-mrgzf","kube-system/global-pull-secret-syncer-xbb69","kube-system/konnectivity-agent-b2sjm","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9","openshift-cluster-node-tuning-operator/tuned-h99nv","openshift-dns/node-resolver-6xvpb","openshift-multus/multus-tm584","openshift-network-diagnostics/network-check-target-v7dql","openshift-ovn-kubernetes/ovnkube-node-fw2m9","kube-system/kube-apiserver-proxy-ip-10-0-138-28.ec2.internal"]
Sep 14 07:48:19.860775 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.860748    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b2sjm"
Sep 14 07:48:19.863079 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.863053    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.863799 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.863776    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 14 07:48:19.863901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.863855    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-n56s4\""
Sep 14 07:48:19.864237 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.864201    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 14 07:48:19.865323 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.865298    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 14 07:48:19.865941 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.865568    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 14 07:48:19.865941 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.865729    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 14 07:48:19.865941 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.865825    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dzrlc\""
Sep 14 07:48:19.866136 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.865993    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 14 07:48:19.866190 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.866160    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:19.866327 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.866307    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 14 07:48:19.867056 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.867039    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 14 07:48:19.871969 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.870080    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 14 07:48:19.871969 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.870281    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 14 07:48:19.871969 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.870484    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 14 07:48:19.871969 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.870691    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 14 07:48:19.871969 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.870889    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 14 07:48:19.871969 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.871105    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fmbvn\""
Sep 14 07:48:19.871969 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.871939    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:19.872829 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:19.872047    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mfj7" podUID="fd5d1c55-8829-4084-9b68-36cac615651b"
Sep 14 07:48:19.872829 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.872403    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mrgzf"
Sep 14 07:48:19.872952 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.872927    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 14 07:48:19.873301 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.873172    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:19.873301 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:19.873250    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbb69" podUID="4b4a6526-1631-4af9-9a20-cf80f53d3eac"
Sep 14 07:48:19.875747 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.875719    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:19.876850 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.876262    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 14 07:48:19.876850 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.876471    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 14 07:48:19.876850 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.876569    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:48:19.876850 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.876676    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-562nk\""
Sep 14 07:48:19.878965 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.878043    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:48:19.878965 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.878524    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pdfcv"
Sep 14 07:48:19.880000 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.879789    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 14 07:48:19.880000 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.879990    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4nmtm\""
Sep 14 07:48:19.882312 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.881136    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:19.882312 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.881693    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 14 07:48:19.882312 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.881873    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jjn58\""
Sep 14 07:48:19.882312 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882012    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 14 07:48:19.882312 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882202    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e941a429-3862-413b-97fc-fceefa08d766-env-overrides\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.882312 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882219    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/db75fca5-ca74-48e0-82f5-ee7514237f7f-host-slash\") pod \"iptables-alerter-mrgzf\" (UID: \"db75fca5-ca74-48e0-82f5-ee7514237f7f\") " pod="openshift-network-operator/iptables-alerter-mrgzf"
Sep 14 07:48:19.882312 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882236    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmhf2\" (UniqueName: \"kubernetes.io/projected/db75fca5-ca74-48e0-82f5-ee7514237f7f-kube-api-access-pmhf2\") pod \"iptables-alerter-mrgzf\" (UID: \"db75fca5-ca74-48e0-82f5-ee7514237f7f\") " pod="openshift-network-operator/iptables-alerter-mrgzf"
Sep 14 07:48:19.882312 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882252    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4b4a6526-1631-4af9-9a20-cf80f53d3eac-kubelet-config\") pod \"global-pull-secret-syncer-xbb69\" (UID: \"4b4a6526-1631-4af9-9a20-cf80f53d3eac\") " pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:19.882312 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882272    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-etc-openvswitch\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.882312 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882304    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-run-ovn\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.882312 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882324    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e941a429-3862-413b-97fc-fceefa08d766-ovnkube-script-lib\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.882884 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882353    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/214873de-7ce7-4bc5-8817-4e82e603f5d2-cni-binary-copy\") pod \"multus-additional-cni-plugins-rbgj4\" (UID: \"214873de-7ce7-4bc5-8817-4e82e603f5d2\") " pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:19.882884 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882380    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/214873de-7ce7-4bc5-8817-4e82e603f5d2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rbgj4\" (UID: \"214873de-7ce7-4bc5-8817-4e82e603f5d2\") " pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:19.882884 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882399    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-run-openvswitch\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.882884 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882404    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 14 07:48:19.882884 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882414    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-systemd-units\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.882884 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882430    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-run-systemd\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.882884 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882452    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-host-cni-netd\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.882884 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882466    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e941a429-3862-413b-97fc-fceefa08d766-ovn-node-metrics-cert\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.882884 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882481    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-node-log\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.882884 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882495    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-host-run-ovn-kubernetes\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.882884 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882524    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.882884 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882549    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret\") pod \"global-pull-secret-syncer-xbb69\" (UID: \"4b4a6526-1631-4af9-9a20-cf80f53d3eac\") " pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:19.882884 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882620    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d4c06442-0a48-45fb-a418-e51fa6a4cea4-agent-certs\") pod \"konnectivity-agent-b2sjm\" (UID: \"d4c06442-0a48-45fb-a418-e51fa6a4cea4\") " pod="kube-system/konnectivity-agent-b2sjm"
Sep 14 07:48:19.882884 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882651    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d4c06442-0a48-45fb-a418-e51fa6a4cea4-konnectivity-ca\") pod \"konnectivity-agent-b2sjm\" (UID: \"d4c06442-0a48-45fb-a418-e51fa6a4cea4\") " pod="kube-system/konnectivity-agent-b2sjm"
Sep 14 07:48:19.882884 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882681    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-host-cni-bin\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.882884 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882713    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/214873de-7ce7-4bc5-8817-4e82e603f5d2-cnibin\") pod \"multus-additional-cni-plugins-rbgj4\" (UID: \"214873de-7ce7-4bc5-8817-4e82e603f5d2\") " pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:19.883695 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882752    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/214873de-7ce7-4bc5-8817-4e82e603f5d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rbgj4\" (UID: \"214873de-7ce7-4bc5-8817-4e82e603f5d2\") " pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:19.883695 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882798    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5xxv\" (UniqueName: \"kubernetes.io/projected/214873de-7ce7-4bc5-8817-4e82e603f5d2-kube-api-access-f5xxv\") pod \"multus-additional-cni-plugins-rbgj4\" (UID: \"214873de-7ce7-4bc5-8817-4e82e603f5d2\") " pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:19.883695 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882831    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-host-slash\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.883695 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882857    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-host-run-netns\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.883695 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882881    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-log-socket\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.883695 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882908    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/214873de-7ce7-4bc5-8817-4e82e603f5d2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rbgj4\" (UID: \"214873de-7ce7-4bc5-8817-4e82e603f5d2\") " pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:19.883695 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882935    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs\") pod \"network-metrics-daemon-4mfj7\" (UID: \"fd5d1c55-8829-4084-9b68-36cac615651b\") " pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:19.883695 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882963    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-var-lib-openvswitch\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.883695 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.882987    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7msw5\" (UniqueName: \"kubernetes.io/projected/e941a429-3862-413b-97fc-fceefa08d766-kube-api-access-7msw5\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.883695 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.883014    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/214873de-7ce7-4bc5-8817-4e82e603f5d2-system-cni-dir\") pod \"multus-additional-cni-plugins-rbgj4\" (UID: \"214873de-7ce7-4bc5-8817-4e82e603f5d2\") " pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:19.883695 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.883050    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25klz\" (UniqueName: \"kubernetes.io/projected/fd5d1c55-8829-4084-9b68-36cac615651b-kube-api-access-25klz\") pod \"network-metrics-daemon-4mfj7\" (UID: \"fd5d1c55-8829-4084-9b68-36cac615651b\") " pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:19.883695 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.883098    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/db75fca5-ca74-48e0-82f5-ee7514237f7f-iptables-alerter-script\") pod \"iptables-alerter-mrgzf\" (UID: \"db75fca5-ca74-48e0-82f5-ee7514237f7f\") " pod="openshift-network-operator/iptables-alerter-mrgzf"
Sep 14 07:48:19.883695 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.883132    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4b4a6526-1631-4af9-9a20-cf80f53d3eac-dbus\") pod \"global-pull-secret-syncer-xbb69\" (UID: \"4b4a6526-1631-4af9-9a20-cf80f53d3eac\") " pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:19.883695 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.883156    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-host-kubelet\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.883695 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.883177    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e941a429-3862-413b-97fc-fceefa08d766-ovnkube-config\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.883695 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.883192    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 14 07:48:19.884462 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.883898    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6xvpb"
Sep 14 07:48:19.884517 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.884465    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 14 07:48:19.885922 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.885386    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mb5cd\""
Sep 14 07:48:19.885922 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.885621    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 14 07:48:19.885922 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.885815    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 14 07:48:19.886173 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.886104    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lr4kj\""
Sep 14 07:48:19.886278 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.886260    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 14 07:48:19.886612 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.886582    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tm584"
Sep 14 07:48:19.890127 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.889900    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 07:48:19.890605 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.890563    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:19.890696 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:19.890669    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7dql" podUID="f3e50313-f82c-47b4-a66b-0bd7f9e227f4"
Sep 14 07:48:19.892254 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.892232    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 14 07:48:19.892543 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.892527    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-w922p\""
Sep 14 07:48:19.910381 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.910355    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-29xvg"
Sep 14 07:48:19.922140 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.922114    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-29xvg"
Sep 14 07:48:19.970884 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.970852    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 14 07:48:19.984471 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.984422    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/214873de-7ce7-4bc5-8817-4e82e603f5d2-cni-binary-copy\") pod \"multus-additional-cni-plugins-rbgj4\" (UID: \"214873de-7ce7-4bc5-8817-4e82e603f5d2\") " pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:19.984471 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.984469    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/214873de-7ce7-4bc5-8817-4e82e603f5d2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rbgj4\" (UID: \"214873de-7ce7-4bc5-8817-4e82e603f5d2\") " pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:19.984707 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.984534    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-var-lib-kubelet\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:19.984707 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.984572    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/59bed5f2-de61-4eae-b5a8-9bebc926467a-registration-dir\") pod \"aws-ebs-csi-driver-node-gr5x9\" (UID: \"59bed5f2-de61-4eae-b5a8-9bebc926467a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:19.984707 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.984600    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/25a3ecfc-dade-4e7c-8347-ce9f629d1f37-hosts-file\") pod \"node-resolver-6xvpb\" (UID: \"25a3ecfc-dade-4e7c-8347-ce9f629d1f37\") " pod="openshift-dns/node-resolver-6xvpb"
Sep 14 07:48:19.984707 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.984626    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-multus-conf-dir\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:19.984707 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.984652    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-etc-sysconfig\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:19.984707 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.984675    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-etc-systemd\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:19.984707 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.984701    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-host-var-lib-cni-multus\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:19.985114 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.984724    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-etc-sysctl-d\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:19.985114 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.984764    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-node-log\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.985114 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.984790    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-host-run-ovn-kubernetes\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.985114 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.984857    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.985114 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.984971    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-node-log\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.985114 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985024    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-host-run-ovn-kubernetes\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.985114 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985080    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret\") pod \"global-pull-secret-syncer-xbb69\" (UID: \"4b4a6526-1631-4af9-9a20-cf80f53d3eac\") " pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:19.985114 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985107    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-hostroot\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:19.985560 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985136    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-etc-kubernetes\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:19.985560 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985162    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f61681e5-1ce0-42ce-8ff4-9394378ede64-tmp\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:19.985560 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985190    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/214873de-7ce7-4bc5-8817-4e82e603f5d2-cni-binary-copy\") pod \"multus-additional-cni-plugins-rbgj4\" (UID: \"214873de-7ce7-4bc5-8817-4e82e603f5d2\") " pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:19.985560 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985204    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77fds\" (UniqueName: \"kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds\") pod \"network-check-target-v7dql\" (UID: \"f3e50313-f82c-47b4-a66b-0bd7f9e227f4\") " pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:19.985560 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985234    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d4c06442-0a48-45fb-a418-e51fa6a4cea4-konnectivity-ca\") pod \"konnectivity-agent-b2sjm\" (UID: \"d4c06442-0a48-45fb-a418-e51fa6a4cea4\") " pod="kube-system/konnectivity-agent-b2sjm"
Sep 14 07:48:19.985560 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985267    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-host-cni-bin\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.985560 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:19.985297    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:19.985560 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985339    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/214873de-7ce7-4bc5-8817-4e82e603f5d2-cnibin\") pod \"multus-additional-cni-plugins-rbgj4\" (UID: \"214873de-7ce7-4bc5-8817-4e82e603f5d2\") " pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:19.985560 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:19.985375    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret podName:4b4a6526-1631-4af9-9a20-cf80f53d3eac nodeName:}" failed. No retries permitted until 2026-09-14 07:48:20.485349233 +0000 UTC m=+2.941799445 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret") pod "global-pull-secret-syncer-xbb69" (UID: "4b4a6526-1631-4af9-9a20-cf80f53d3eac") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:19.985560 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985303    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/214873de-7ce7-4bc5-8817-4e82e603f5d2-cnibin\") pod \"multus-additional-cni-plugins-rbgj4\" (UID: \"214873de-7ce7-4bc5-8817-4e82e603f5d2\") " pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:19.985560 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985394    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/214873de-7ce7-4bc5-8817-4e82e603f5d2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rbgj4\" (UID: \"214873de-7ce7-4bc5-8817-4e82e603f5d2\") " pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:19.985560 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985415    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-cnibin\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:19.985560 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985445    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-host-run-netns\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:19.985560 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985451    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.985560 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985383    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-host-cni-bin\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.985560 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985490    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-log-socket\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.985560 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985518    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-multus-cni-dir\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:19.986483 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985546    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-lib-modules\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:19.986483 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985576    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-log-socket\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.986483 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985572    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/214873de-7ce7-4bc5-8817-4e82e603f5d2-system-cni-dir\") pod \"multus-additional-cni-plugins-rbgj4\" (UID: \"214873de-7ce7-4bc5-8817-4e82e603f5d2\") " pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:19.986483 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985627    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/214873de-7ce7-4bc5-8817-4e82e603f5d2-system-cni-dir\") pod \"multus-additional-cni-plugins-rbgj4\" (UID: \"214873de-7ce7-4bc5-8817-4e82e603f5d2\") " pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:19.986483 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985627    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25klz\" (UniqueName: \"kubernetes.io/projected/fd5d1c55-8829-4084-9b68-36cac615651b-kube-api-access-25klz\") pod \"network-metrics-daemon-4mfj7\" (UID: \"fd5d1c55-8829-4084-9b68-36cac615651b\") " pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:19.986483 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985666    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4b4a6526-1631-4af9-9a20-cf80f53d3eac-dbus\") pod \"global-pull-secret-syncer-xbb69\" (UID: \"4b4a6526-1631-4af9-9a20-cf80f53d3eac\") " pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:19.986483 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985728    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4079005f-d292-4b05-9d42-471d08e7d92a-cni-binary-copy\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:19.986483 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985754    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-etc-kubernetes\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:19.986483 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985779    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-host-kubelet\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.986483 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985796    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4b4a6526-1631-4af9-9a20-cf80f53d3eac-dbus\") pod \"global-pull-secret-syncer-xbb69\" (UID: \"4b4a6526-1631-4af9-9a20-cf80f53d3eac\") " pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:19.986483 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985803    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e941a429-3862-413b-97fc-fceefa08d766-ovnkube-config\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.986483 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985838    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e941a429-3862-413b-97fc-fceefa08d766-env-overrides\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.986483 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985867    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/db75fca5-ca74-48e0-82f5-ee7514237f7f-host-slash\") pod \"iptables-alerter-mrgzf\" (UID: \"db75fca5-ca74-48e0-82f5-ee7514237f7f\") " pod="openshift-network-operator/iptables-alerter-mrgzf"
Sep 14 07:48:19.986483 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985898    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4b4a6526-1631-4af9-9a20-cf80f53d3eac-kubelet-config\") pod \"global-pull-secret-syncer-xbb69\" (UID: \"4b4a6526-1631-4af9-9a20-cf80f53d3eac\") " pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:19.986483 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985925    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/59bed5f2-de61-4eae-b5a8-9bebc926467a-socket-dir\") pod \"aws-ebs-csi-driver-node-gr5x9\" (UID: \"59bed5f2-de61-4eae-b5a8-9bebc926467a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:19.986483 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985951    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/59bed5f2-de61-4eae-b5a8-9bebc926467a-device-dir\") pod \"aws-ebs-csi-driver-node-gr5x9\" (UID: \"59bed5f2-de61-4eae-b5a8-9bebc926467a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:19.986483 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.985977    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9zjh\" (UniqueName: \"kubernetes.io/projected/f61681e5-1ce0-42ce-8ff4-9394378ede64-kube-api-access-z9zjh\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:19.987361 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986005    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-etc-openvswitch\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.987361 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986030    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-system-cni-dir\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:19.987361 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986057    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-sys\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:19.987361 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986097    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-host\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:19.987361 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986127    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9594803d-c72d-4f5a-9b8d-e78d40e17d92-host\") pod \"node-ca-pdfcv\" (UID: \"9594803d-c72d-4f5a-9b8d-e78d40e17d92\") " pod="openshift-image-registry/node-ca-pdfcv"
Sep 14 07:48:19.987361 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986162    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-run-openvswitch\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.987361 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986190    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsn55\" (UniqueName: \"kubernetes.io/projected/59bed5f2-de61-4eae-b5a8-9bebc926467a-kube-api-access-hsn55\") pod \"aws-ebs-csi-driver-node-gr5x9\" (UID: \"59bed5f2-de61-4eae-b5a8-9bebc926467a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:19.987361 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986232    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mjkn\" (UniqueName: \"kubernetes.io/projected/25a3ecfc-dade-4e7c-8347-ce9f629d1f37-kube-api-access-8mjkn\") pod \"node-resolver-6xvpb\" (UID: \"25a3ecfc-dade-4e7c-8347-ce9f629d1f37\") " pod="openshift-dns/node-resolver-6xvpb"
Sep 14 07:48:19.987361 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986260    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-systemd-units\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.987361 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986284    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-run-systemd\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.987361 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986297    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e941a429-3862-413b-97fc-fceefa08d766-env-overrides\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.987361 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986309    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-host-cni-netd\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.987361 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986351    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e941a429-3862-413b-97fc-fceefa08d766-ovn-node-metrics-cert\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.987361 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986376    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e941a429-3862-413b-97fc-fceefa08d766-ovnkube-config\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.987361 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986386    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/db75fca5-ca74-48e0-82f5-ee7514237f7f-host-slash\") pod \"iptables-alerter-mrgzf\" (UID: \"db75fca5-ca74-48e0-82f5-ee7514237f7f\") " pod="openshift-network-operator/iptables-alerter-mrgzf"
Sep 14 07:48:19.987361 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986378    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/59bed5f2-de61-4eae-b5a8-9bebc926467a-etc-selinux\") pod \"aws-ebs-csi-driver-node-gr5x9\" (UID: \"59bed5f2-de61-4eae-b5a8-9bebc926467a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:19.987361 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986436    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d4c06442-0a48-45fb-a418-e51fa6a4cea4-konnectivity-ca\") pod \"konnectivity-agent-b2sjm\" (UID: \"d4c06442-0a48-45fb-a418-e51fa6a4cea4\") " pod="kube-system/konnectivity-agent-b2sjm"
Sep 14 07:48:19.988223 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986451    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f61681e5-1ce0-42ce-8ff4-9394378ede64-etc-tuned\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:19.988223 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986440    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4b4a6526-1631-4af9-9a20-cf80f53d3eac-kubelet-config\") pod \"global-pull-secret-syncer-xbb69\" (UID: \"4b4a6526-1631-4af9-9a20-cf80f53d3eac\") " pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:19.988223 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986462    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-systemd-units\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.988223 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986493    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-host-kubelet\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.988223 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986503    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-run-openvswitch\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.988223 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986508    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-etc-openvswitch\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.988223 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986529    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-run-systemd\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.988223 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986569    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-multus-socket-dir-parent\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:19.988223 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986613    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-host-cni-netd\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.988223 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986752    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4079005f-d292-4b05-9d42-471d08e7d92a-multus-daemon-config\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:19.988223 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986791    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-run\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:19.988223 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986837    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d4c06442-0a48-45fb-a418-e51fa6a4cea4-agent-certs\") pod \"konnectivity-agent-b2sjm\" (UID: \"d4c06442-0a48-45fb-a418-e51fa6a4cea4\") " pod="kube-system/konnectivity-agent-b2sjm"
Sep 14 07:48:19.988223 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986945    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/214873de-7ce7-4bc5-8817-4e82e603f5d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rbgj4\" (UID: \"214873de-7ce7-4bc5-8817-4e82e603f5d2\") " pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:19.988223 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.986977    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5xxv\" (UniqueName: \"kubernetes.io/projected/214873de-7ce7-4bc5-8817-4e82e603f5d2-kube-api-access-f5xxv\") pod \"multus-additional-cni-plugins-rbgj4\" (UID: \"214873de-7ce7-4bc5-8817-4e82e603f5d2\") " pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:19.988223 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987024    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bcg7\" (UniqueName: \"kubernetes.io/projected/4079005f-d292-4b05-9d42-471d08e7d92a-kube-api-access-6bcg7\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:19.988223 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987054    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-host-slash\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.988223 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987161    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-host-run-netns\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.989091 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987192    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/214873de-7ce7-4bc5-8817-4e82e603f5d2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rbgj4\" (UID: \"214873de-7ce7-4bc5-8817-4e82e603f5d2\") " pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:19.989091 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987220    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs\") pod \"network-metrics-daemon-4mfj7\" (UID: \"fd5d1c55-8829-4084-9b68-36cac615651b\") " pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:19.989091 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987276    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-host-slash\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.989091 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987304    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-host-run-k8s-cni-cncf-io\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:19.989091 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987352    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-etc-modprobe-d\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:19.989091 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987353    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 14 07:48:19.989091 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987383    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9fbc\" (UniqueName: \"kubernetes.io/projected/9594803d-c72d-4f5a-9b8d-e78d40e17d92-kube-api-access-c9fbc\") pod \"node-ca-pdfcv\" (UID: \"9594803d-c72d-4f5a-9b8d-e78d40e17d92\") " pod="openshift-image-registry/node-ca-pdfcv"
Sep 14 07:48:19.989091 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987384    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/214873de-7ce7-4bc5-8817-4e82e603f5d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rbgj4\" (UID: \"214873de-7ce7-4bc5-8817-4e82e603f5d2\") " pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:19.989091 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987431    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-var-lib-openvswitch\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.989091 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987455    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-host-run-netns\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.989091 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987463    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7msw5\" (UniqueName: \"kubernetes.io/projected/e941a429-3862-413b-97fc-fceefa08d766-kube-api-access-7msw5\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.989091 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987499    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/db75fca5-ca74-48e0-82f5-ee7514237f7f-iptables-alerter-script\") pod \"iptables-alerter-mrgzf\" (UID: \"db75fca5-ca74-48e0-82f5-ee7514237f7f\") " pod="openshift-network-operator/iptables-alerter-mrgzf"
Sep 14 07:48:19.989091 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987508    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-var-lib-openvswitch\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.989091 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987551    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/59bed5f2-de61-4eae-b5a8-9bebc926467a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gr5x9\" (UID: \"59bed5f2-de61-4eae-b5a8-9bebc926467a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:19.989091 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:19.987642    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:19.989091 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987736    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/214873de-7ce7-4bc5-8817-4e82e603f5d2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rbgj4\" (UID: \"214873de-7ce7-4bc5-8817-4e82e603f5d2\") " pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:19.989091 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987746    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-host-var-lib-cni-bin\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:19.989983 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:19.987832    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs podName:fd5d1c55-8829-4084-9b68-36cac615651b nodeName:}" failed. No retries permitted until 2026-09-14 07:48:20.487813471 +0000 UTC m=+2.944263452 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs") pod "network-metrics-daemon-4mfj7" (UID: "fd5d1c55-8829-4084-9b68-36cac615651b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:19.989983 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987855    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-host-var-lib-kubelet\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:19.989983 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987900    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-host-run-multus-certs\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:19.989983 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987930    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-etc-sysctl-conf\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:19.989983 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.987969    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmhf2\" (UniqueName: \"kubernetes.io/projected/db75fca5-ca74-48e0-82f5-ee7514237f7f-kube-api-access-pmhf2\") pod \"iptables-alerter-mrgzf\" (UID: \"db75fca5-ca74-48e0-82f5-ee7514237f7f\") " pod="openshift-network-operator/iptables-alerter-mrgzf"
Sep 14 07:48:19.989983 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.988028    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/59bed5f2-de61-4eae-b5a8-9bebc926467a-sys-fs\") pod \"aws-ebs-csi-driver-node-gr5x9\" (UID: \"59bed5f2-de61-4eae-b5a8-9bebc926467a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:19.989983 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.988059    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25a3ecfc-dade-4e7c-8347-ce9f629d1f37-tmp-dir\") pod \"node-resolver-6xvpb\" (UID: \"25a3ecfc-dade-4e7c-8347-ce9f629d1f37\") " pod="openshift-dns/node-resolver-6xvpb"
Sep 14 07:48:19.989983 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.988093    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9594803d-c72d-4f5a-9b8d-e78d40e17d92-serviceca\") pod \"node-ca-pdfcv\" (UID: \"9594803d-c72d-4f5a-9b8d-e78d40e17d92\") " pod="openshift-image-registry/node-ca-pdfcv"
Sep 14 07:48:19.989983 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.988122    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-run-ovn\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.989983 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.988168    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e941a429-3862-413b-97fc-fceefa08d766-ovnkube-script-lib\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.989983 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.988208    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/db75fca5-ca74-48e0-82f5-ee7514237f7f-iptables-alerter-script\") pod \"iptables-alerter-mrgzf\" (UID: \"db75fca5-ca74-48e0-82f5-ee7514237f7f\") " pod="openshift-network-operator/iptables-alerter-mrgzf"
Sep 14 07:48:19.989983 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.988329    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e941a429-3862-413b-97fc-fceefa08d766-run-ovn\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.989983 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.988813    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e941a429-3862-413b-97fc-fceefa08d766-ovnkube-script-lib\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.995537 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.995478    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e941a429-3862-413b-97fc-fceefa08d766-ovn-node-metrics-cert\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:19.995724 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.995710    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d4c06442-0a48-45fb-a418-e51fa6a4cea4-agent-certs\") pod \"konnectivity-agent-b2sjm\" (UID: \"d4c06442-0a48-45fb-a418-e51fa6a4cea4\") " pod="kube-system/konnectivity-agent-b2sjm"
Sep 14 07:48:19.997875 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.997851    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25klz\" (UniqueName: \"kubernetes.io/projected/fd5d1c55-8829-4084-9b68-36cac615651b-kube-api-access-25klz\") pod \"network-metrics-daemon-4mfj7\" (UID: \"fd5d1c55-8829-4084-9b68-36cac615651b\") " pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:19.999777 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:19.999748    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmhf2\" (UniqueName: \"kubernetes.io/projected/db75fca5-ca74-48e0-82f5-ee7514237f7f-kube-api-access-pmhf2\") pod \"iptables-alerter-mrgzf\" (UID: \"db75fca5-ca74-48e0-82f5-ee7514237f7f\") " pod="openshift-network-operator/iptables-alerter-mrgzf"
Sep 14 07:48:20.000536 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.000511    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5xxv\" (UniqueName: \"kubernetes.io/projected/214873de-7ce7-4bc5-8817-4e82e603f5d2-kube-api-access-f5xxv\") pod \"multus-additional-cni-plugins-rbgj4\" (UID: \"214873de-7ce7-4bc5-8817-4e82e603f5d2\") " pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:20.001821 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.001790    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7msw5\" (UniqueName: \"kubernetes.io/projected/e941a429-3862-413b-97fc-fceefa08d766-kube-api-access-7msw5\") pod \"ovnkube-node-fw2m9\" (UID: \"e941a429-3862-413b-97fc-fceefa08d766\") " pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:20.088707 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.088671    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/59bed5f2-de61-4eae-b5a8-9bebc926467a-socket-dir\") pod \"aws-ebs-csi-driver-node-gr5x9\" (UID: \"59bed5f2-de61-4eae-b5a8-9bebc926467a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:20.088880 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.088785    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/59bed5f2-de61-4eae-b5a8-9bebc926467a-socket-dir\") pod \"aws-ebs-csi-driver-node-gr5x9\" (UID: \"59bed5f2-de61-4eae-b5a8-9bebc926467a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:20.088880 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.088839    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/59bed5f2-de61-4eae-b5a8-9bebc926467a-device-dir\") pod \"aws-ebs-csi-driver-node-gr5x9\" (UID: \"59bed5f2-de61-4eae-b5a8-9bebc926467a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:20.088880 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.088874    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9zjh\" (UniqueName: \"kubernetes.io/projected/f61681e5-1ce0-42ce-8ff4-9394378ede64-kube-api-access-z9zjh\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.089123 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.088902    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-system-cni-dir\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.089123 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.088961    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-sys\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.089123 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089013    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-sys\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.089123 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089033    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-host\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.089123 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089083    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-host\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.089123 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089082    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-system-cni-dir\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.089123 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089103    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9594803d-c72d-4f5a-9b8d-e78d40e17d92-host\") pod \"node-ca-pdfcv\" (UID: \"9594803d-c72d-4f5a-9b8d-e78d40e17d92\") " pod="openshift-image-registry/node-ca-pdfcv"
Sep 14 07:48:20.089123 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089125    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsn55\" (UniqueName: \"kubernetes.io/projected/59bed5f2-de61-4eae-b5a8-9bebc926467a-kube-api-access-hsn55\") pod \"aws-ebs-csi-driver-node-gr5x9\" (UID: \"59bed5f2-de61-4eae-b5a8-9bebc926467a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:20.089123 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089130    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9594803d-c72d-4f5a-9b8d-e78d40e17d92-host\") pod \"node-ca-pdfcv\" (UID: \"9594803d-c72d-4f5a-9b8d-e78d40e17d92\") " pod="openshift-image-registry/node-ca-pdfcv"
Sep 14 07:48:20.089572 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.088960    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/59bed5f2-de61-4eae-b5a8-9bebc926467a-device-dir\") pod \"aws-ebs-csi-driver-node-gr5x9\" (UID: \"59bed5f2-de61-4eae-b5a8-9bebc926467a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:20.089572 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089140    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mjkn\" (UniqueName: \"kubernetes.io/projected/25a3ecfc-dade-4e7c-8347-ce9f629d1f37-kube-api-access-8mjkn\") pod \"node-resolver-6xvpb\" (UID: \"25a3ecfc-dade-4e7c-8347-ce9f629d1f37\") " pod="openshift-dns/node-resolver-6xvpb"
Sep 14 07:48:20.089572 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089160    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/59bed5f2-de61-4eae-b5a8-9bebc926467a-etc-selinux\") pod \"aws-ebs-csi-driver-node-gr5x9\" (UID: \"59bed5f2-de61-4eae-b5a8-9bebc926467a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:20.089572 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089182    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f61681e5-1ce0-42ce-8ff4-9394378ede64-etc-tuned\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.089572 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089267    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/59bed5f2-de61-4eae-b5a8-9bebc926467a-etc-selinux\") pod \"aws-ebs-csi-driver-node-gr5x9\" (UID: \"59bed5f2-de61-4eae-b5a8-9bebc926467a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:20.089572 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089295    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-multus-socket-dir-parent\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.089572 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089327    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4079005f-d292-4b05-9d42-471d08e7d92a-multus-daemon-config\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.089572 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089351    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-run\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.089572 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089381    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bcg7\" (UniqueName: \"kubernetes.io/projected/4079005f-d292-4b05-9d42-471d08e7d92a-kube-api-access-6bcg7\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.089572 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089397    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-multus-socket-dir-parent\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.089572 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089422    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-host-run-k8s-cni-cncf-io\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.089572 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089434    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-run\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.089572 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089444    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-etc-modprobe-d\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.089572 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089492    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9fbc\" (UniqueName: \"kubernetes.io/projected/9594803d-c72d-4f5a-9b8d-e78d40e17d92-kube-api-access-c9fbc\") pod \"node-ca-pdfcv\" (UID: \"9594803d-c72d-4f5a-9b8d-e78d40e17d92\") " pod="openshift-image-registry/node-ca-pdfcv"
Sep 14 07:48:20.089572 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089555    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/59bed5f2-de61-4eae-b5a8-9bebc926467a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gr5x9\" (UID: \"59bed5f2-de61-4eae-b5a8-9bebc926467a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:20.089572 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089581    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-host-var-lib-cni-bin\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.089572 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089581    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-etc-modprobe-d\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.090495 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089664    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/59bed5f2-de61-4eae-b5a8-9bebc926467a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gr5x9\" (UID: \"59bed5f2-de61-4eae-b5a8-9bebc926467a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:20.090495 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089665    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-host-var-lib-cni-bin\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.090495 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089664    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-host-run-k8s-cni-cncf-io\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.090495 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089719    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-host-var-lib-kubelet\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.090495 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089751    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-host-run-multus-certs\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.090495 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089783    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-etc-sysctl-conf\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.090495 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089789    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-host-run-multus-certs\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.090495 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089790    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-host-var-lib-kubelet\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.090495 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089809    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/59bed5f2-de61-4eae-b5a8-9bebc926467a-sys-fs\") pod \"aws-ebs-csi-driver-node-gr5x9\" (UID: \"59bed5f2-de61-4eae-b5a8-9bebc926467a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:20.090495 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089849    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25a3ecfc-dade-4e7c-8347-ce9f629d1f37-tmp-dir\") pod \"node-resolver-6xvpb\" (UID: \"25a3ecfc-dade-4e7c-8347-ce9f629d1f37\") " pod="openshift-dns/node-resolver-6xvpb"
Sep 14 07:48:20.090495 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089856    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/59bed5f2-de61-4eae-b5a8-9bebc926467a-sys-fs\") pod \"aws-ebs-csi-driver-node-gr5x9\" (UID: \"59bed5f2-de61-4eae-b5a8-9bebc926467a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:20.090495 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089902    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9594803d-c72d-4f5a-9b8d-e78d40e17d92-serviceca\") pod \"node-ca-pdfcv\" (UID: \"9594803d-c72d-4f5a-9b8d-e78d40e17d92\") " pod="openshift-image-registry/node-ca-pdfcv"
Sep 14 07:48:20.090495 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089947    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-etc-sysctl-conf\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.090495 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089964    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-var-lib-kubelet\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.090495 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.089983    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/59bed5f2-de61-4eae-b5a8-9bebc926467a-registration-dir\") pod \"aws-ebs-csi-driver-node-gr5x9\" (UID: \"59bed5f2-de61-4eae-b5a8-9bebc926467a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:20.090495 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090002    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/25a3ecfc-dade-4e7c-8347-ce9f629d1f37-hosts-file\") pod \"node-resolver-6xvpb\" (UID: \"25a3ecfc-dade-4e7c-8347-ce9f629d1f37\") " pod="openshift-dns/node-resolver-6xvpb"
Sep 14 07:48:20.090495 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090047    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-multus-conf-dir\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.090495 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090072    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-etc-sysconfig\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.091512 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090099    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-etc-systemd\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.091512 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090117    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/59bed5f2-de61-4eae-b5a8-9bebc926467a-registration-dir\") pod \"aws-ebs-csi-driver-node-gr5x9\" (UID: \"59bed5f2-de61-4eae-b5a8-9bebc926467a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:20.091512 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090124    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-host-var-lib-cni-multus\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.091512 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090156    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-var-lib-kubelet\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.091512 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090173    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-host-var-lib-cni-multus\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.091512 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090174    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-etc-sysctl-d\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.091512 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090208    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25a3ecfc-dade-4e7c-8347-ce9f629d1f37-tmp-dir\") pod \"node-resolver-6xvpb\" (UID: \"25a3ecfc-dade-4e7c-8347-ce9f629d1f37\") " pod="openshift-dns/node-resolver-6xvpb"
Sep 14 07:48:20.091512 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090225    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-hostroot\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.091512 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090267    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-etc-sysconfig\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.091512 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090225    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/25a3ecfc-dade-4e7c-8347-ce9f629d1f37-hosts-file\") pod \"node-resolver-6xvpb\" (UID: \"25a3ecfc-dade-4e7c-8347-ce9f629d1f37\") " pod="openshift-dns/node-resolver-6xvpb"
Sep 14 07:48:20.091512 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090270    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-etc-kubernetes\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.091512 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090275    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-etc-sysctl-d\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.091512 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090298    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-etc-systemd\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.091512 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090309    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-multus-conf-dir\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.091512 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090314    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f61681e5-1ce0-42ce-8ff4-9394378ede64-tmp\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.091512 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090340    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-etc-kubernetes\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.091512 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090340    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4079005f-d292-4b05-9d42-471d08e7d92a-multus-daemon-config\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.091512 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090348    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77fds\" (UniqueName: \"kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds\") pod \"network-check-target-v7dql\" (UID: \"f3e50313-f82c-47b4-a66b-0bd7f9e227f4\") " pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:20.092620 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090365    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9594803d-c72d-4f5a-9b8d-e78d40e17d92-serviceca\") pod \"node-ca-pdfcv\" (UID: \"9594803d-c72d-4f5a-9b8d-e78d40e17d92\") " pod="openshift-image-registry/node-ca-pdfcv"
Sep 14 07:48:20.092620 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090380    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-hostroot\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.092620 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090382    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-cnibin\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.092620 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090492    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-host-run-netns\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.092620 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090541    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-multus-cni-dir\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.092620 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090567    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-lib-modules\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.092620 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090563    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-cnibin\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.092620 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090607    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4079005f-d292-4b05-9d42-471d08e7d92a-cni-binary-copy\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.092620 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090611    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-host-run-netns\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.092620 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090635    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-etc-kubernetes\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.092620 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090645    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-multus-cni-dir\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.092620 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090687    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f61681e5-1ce0-42ce-8ff4-9394378ede64-lib-modules\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.092620 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.090722    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4079005f-d292-4b05-9d42-471d08e7d92a-etc-kubernetes\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.092620 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.091120    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4079005f-d292-4b05-9d42-471d08e7d92a-cni-binary-copy\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.093313 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.092784    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f61681e5-1ce0-42ce-8ff4-9394378ede64-etc-tuned\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.093463 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.093444    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f61681e5-1ce0-42ce-8ff4-9394378ede64-tmp\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.100003 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:20.099977    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:48:20.100145 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:20.100008    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:48:20.100145 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:20.100023    2562 projected.go:196] Error preparing data for projected volume kube-api-access-77fds for pod openshift-network-diagnostics/network-check-target-v7dql: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:20.100145 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:20.100100    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds podName:f3e50313-f82c-47b4-a66b-0bd7f9e227f4 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:20.600080556 +0000 UTC m=+3.056530549 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-77fds" (UniqueName: "kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds") pod "network-check-target-v7dql" (UID: "f3e50313-f82c-47b4-a66b-0bd7f9e227f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:20.101477 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.101442    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsn55\" (UniqueName: \"kubernetes.io/projected/59bed5f2-de61-4eae-b5a8-9bebc926467a-kube-api-access-hsn55\") pod \"aws-ebs-csi-driver-node-gr5x9\" (UID: \"59bed5f2-de61-4eae-b5a8-9bebc926467a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:20.103360 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.103336    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9fbc\" (UniqueName: \"kubernetes.io/projected/9594803d-c72d-4f5a-9b8d-e78d40e17d92-kube-api-access-c9fbc\") pod \"node-ca-pdfcv\" (UID: \"9594803d-c72d-4f5a-9b8d-e78d40e17d92\") " pod="openshift-image-registry/node-ca-pdfcv"
Sep 14 07:48:20.103627 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.103605    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9zjh\" (UniqueName: \"kubernetes.io/projected/f61681e5-1ce0-42ce-8ff4-9394378ede64-kube-api-access-z9zjh\") pod \"tuned-h99nv\" (UID: \"f61681e5-1ce0-42ce-8ff4-9394378ede64\") " pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.104661 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.104608    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mjkn\" (UniqueName: \"kubernetes.io/projected/25a3ecfc-dade-4e7c-8347-ce9f629d1f37-kube-api-access-8mjkn\") pod \"node-resolver-6xvpb\" (UID: \"25a3ecfc-dade-4e7c-8347-ce9f629d1f37\") " pod="openshift-dns/node-resolver-6xvpb"
Sep 14 07:48:20.105529 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.105506    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bcg7\" (UniqueName: \"kubernetes.io/projected/4079005f-d292-4b05-9d42-471d08e7d92a-kube-api-access-6bcg7\") pod \"multus-tm584\" (UID: \"4079005f-d292-4b05-9d42-471d08e7d92a\") " pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.111780 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.111737    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-28.ec2.internal" event={"ID":"ef4817a2551205637f03cf112a16afcb","Type":"ContainerStarted","Data":"8b6eda20d5b09670fb3f9275eca4ed35dedc63a2bd111925741637dbe288bb9a"}
Sep 14 07:48:20.113654 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.113627    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-28.ec2.internal" event={"ID":"eb7e8b3289b4b5c21abcc1b439d2f0dd","Type":"ContainerStarted","Data":"d9ae53b099aed7fc94725aa3daa392f6e647d7e5bb4d63c4421785f427ac51cb"}
Sep 14 07:48:20.178113 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.178073    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b2sjm"
Sep 14 07:48:20.191896 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.191864    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:20.205128 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.205058    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rbgj4"
Sep 14 07:48:20.210032 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.210007    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mrgzf"
Sep 14 07:48:20.217978 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.217946    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h99nv"
Sep 14 07:48:20.226534 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.226495    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pdfcv"
Sep 14 07:48:20.236564 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.236537    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9"
Sep 14 07:48:20.245307 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.245283    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6xvpb"
Sep 14 07:48:20.254282 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.254258    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tm584"
Sep 14 07:48:20.494314 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.494236    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs\") pod \"network-metrics-daemon-4mfj7\" (UID: \"fd5d1c55-8829-4084-9b68-36cac615651b\") " pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:20.494314 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.494294    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret\") pod \"global-pull-secret-syncer-xbb69\" (UID: \"4b4a6526-1631-4af9-9a20-cf80f53d3eac\") " pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:20.494488 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:20.494407    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:20.494488 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:20.494473    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs podName:fd5d1c55-8829-4084-9b68-36cac615651b nodeName:}" failed. No retries permitted until 2026-09-14 07:48:21.494453154 +0000 UTC m=+3.950903137 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs") pod "network-metrics-daemon-4mfj7" (UID: "fd5d1c55-8829-4084-9b68-36cac615651b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:20.494617 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:20.494520    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:20.494617 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:20.494578    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret podName:4b4a6526-1631-4af9-9a20-cf80f53d3eac nodeName:}" failed. No retries permitted until 2026-09-14 07:48:21.494562462 +0000 UTC m=+3.951012456 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret") pod "global-pull-secret-syncer-xbb69" (UID: "4b4a6526-1631-4af9-9a20-cf80f53d3eac") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:20.696453 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.696420    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77fds\" (UniqueName: \"kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds\") pod \"network-check-target-v7dql\" (UID: \"f3e50313-f82c-47b4-a66b-0bd7f9e227f4\") " pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:20.696634 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:20.696569    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:48:20.696634 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:20.696589    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:48:20.696634 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:20.696603    2562 projected.go:196] Error preparing data for projected volume kube-api-access-77fds for pod openshift-network-diagnostics/network-check-target-v7dql: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:20.696753 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:20.696667    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds podName:f3e50313-f82c-47b4-a66b-0bd7f9e227f4 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:21.696653229 +0000 UTC m=+4.153103206 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-77fds" (UniqueName: "kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds") pod "network-check-target-v7dql" (UID: "f3e50313-f82c-47b4-a66b-0bd7f9e227f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:20.926736 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.926639    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 07:43:20 +0000 UTC" deadline="2028-02-08 15:21:38.46535805 +0000 UTC"
Sep 14 07:48:20.926736 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:20.926680    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12295h33m17.538685055s"
Sep 14 07:48:21.106315 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:21.106277    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:21.106510 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:21.106277    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:21.106510 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:21.106404    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbb69" podUID="4b4a6526-1631-4af9-9a20-cf80f53d3eac"
Sep 14 07:48:21.106510 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:21.106472    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mfj7" podUID="fd5d1c55-8829-4084-9b68-36cac615651b"
Sep 14 07:48:21.207431 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:21.207269    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4c06442_0a48_45fb_a418_e51fa6a4cea4.slice/crio-71da0ff5c2f26f40aa4dcade6e239e78f3ae9857787098fd4968052193afe167 WatchSource:0}: Error finding container 71da0ff5c2f26f40aa4dcade6e239e78f3ae9857787098fd4968052193afe167: Status 404 returned error can't find the container with id 71da0ff5c2f26f40aa4dcade6e239e78f3ae9857787098fd4968052193afe167
Sep 14 07:48:21.208668 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:21.208644    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4079005f_d292_4b05_9d42_471d08e7d92a.slice/crio-ec93b8ed6d049c328ccacc2558f4182df20e7c7c4dd4d0f97f4504dbae226d39 WatchSource:0}: Error finding container ec93b8ed6d049c328ccacc2558f4182df20e7c7c4dd4d0f97f4504dbae226d39: Status 404 returned error can't find the container with id ec93b8ed6d049c328ccacc2558f4182df20e7c7c4dd4d0f97f4504dbae226d39
Sep 14 07:48:21.209277 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:21.209254    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25a3ecfc_dade_4e7c_8347_ce9f629d1f37.slice/crio-8d0fe8f86c66aff9d93c38fca54227d17f97f4405473fd3661fc0b7a6f861be9 WatchSource:0}: Error finding container 8d0fe8f86c66aff9d93c38fca54227d17f97f4405473fd3661fc0b7a6f861be9: Status 404 returned error can't find the container with id 8d0fe8f86c66aff9d93c38fca54227d17f97f4405473fd3661fc0b7a6f861be9
Sep 14 07:48:21.216208 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:21.216145    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod214873de_7ce7_4bc5_8817_4e82e603f5d2.slice/crio-4220a1520946746d641baf24331fa9f88f0ce37f4b91a3360cce18caf4084eae WatchSource:0}: Error finding container 4220a1520946746d641baf24331fa9f88f0ce37f4b91a3360cce18caf4084eae: Status 404 returned error can't find the container with id 4220a1520946746d641baf24331fa9f88f0ce37f4b91a3360cce18caf4084eae
Sep 14 07:48:21.217206 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:21.217180    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9594803d_c72d_4f5a_9b8d_e78d40e17d92.slice/crio-bb7cb1e0471e28744458402b8989d1a369e05c512f8a067c17e226d41c84798e WatchSource:0}: Error finding container bb7cb1e0471e28744458402b8989d1a369e05c512f8a067c17e226d41c84798e: Status 404 returned error can't find the container with id bb7cb1e0471e28744458402b8989d1a369e05c512f8a067c17e226d41c84798e
Sep 14 07:48:21.218183 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:21.218161    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb75fca5_ca74_48e0_82f5_ee7514237f7f.slice/crio-987f97fd13cbd5dede59d50a53dd1869891bb9b1830058799073a5e9985a3cec WatchSource:0}: Error finding container 987f97fd13cbd5dede59d50a53dd1869891bb9b1830058799073a5e9985a3cec: Status 404 returned error can't find the container with id 987f97fd13cbd5dede59d50a53dd1869891bb9b1830058799073a5e9985a3cec
Sep 14 07:48:21.219875 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:21.219848    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59bed5f2_de61_4eae_b5a8_9bebc926467a.slice/crio-be824871f2d4abfb8ebd8b871f29978af0acd4b7fad67d60a9be50d9b27e8e7a WatchSource:0}: Error finding container be824871f2d4abfb8ebd8b871f29978af0acd4b7fad67d60a9be50d9b27e8e7a: Status 404 returned error can't find the container with id be824871f2d4abfb8ebd8b871f29978af0acd4b7fad67d60a9be50d9b27e8e7a
Sep 14 07:48:21.503560 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:21.503527    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret\") pod \"global-pull-secret-syncer-xbb69\" (UID: \"4b4a6526-1631-4af9-9a20-cf80f53d3eac\") " pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:21.503685 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:21.503600    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs\") pod \"network-metrics-daemon-4mfj7\" (UID: \"fd5d1c55-8829-4084-9b68-36cac615651b\") " pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:21.503744 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:21.503677    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:21.503744 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:21.503692    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:21.503744 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:21.503735    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret podName:4b4a6526-1631-4af9-9a20-cf80f53d3eac nodeName:}" failed. No retries permitted until 2026-09-14 07:48:23.503719262 +0000 UTC m=+5.960169240 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret") pod "global-pull-secret-syncer-xbb69" (UID: "4b4a6526-1631-4af9-9a20-cf80f53d3eac") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:21.503923 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:21.503751    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs podName:fd5d1c55-8829-4084-9b68-36cac615651b nodeName:}" failed. No retries permitted until 2026-09-14 07:48:23.503744248 +0000 UTC m=+5.960194220 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs") pod "network-metrics-daemon-4mfj7" (UID: "fd5d1c55-8829-4084-9b68-36cac615651b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:21.704915 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:21.704877    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77fds\" (UniqueName: \"kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds\") pod \"network-check-target-v7dql\" (UID: \"f3e50313-f82c-47b4-a66b-0bd7f9e227f4\") " pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:21.705102 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:21.705075    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:48:21.705102 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:21.705100    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:48:21.705221 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:21.705114    2562 projected.go:196] Error preparing data for projected volume kube-api-access-77fds for pod openshift-network-diagnostics/network-check-target-v7dql: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:21.705221 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:21.705186    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds podName:f3e50313-f82c-47b4-a66b-0bd7f9e227f4 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:23.70516707 +0000 UTC m=+6.161617048 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-77fds" (UniqueName: "kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds") pod "network-check-target-v7dql" (UID: "f3e50313-f82c-47b4-a66b-0bd7f9e227f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:21.927855 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:21.927735    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 07:43:20 +0000 UTC" deadline="2028-06-18 09:41:10.397311504 +0000 UTC"
Sep 14 07:48:21.927855 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:21.927802    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15433h52m48.469514374s"
Sep 14 07:48:22.110276 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:22.110240    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:22.110442 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:22.110385    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7dql" podUID="f3e50313-f82c-47b4-a66b-0bd7f9e227f4"
Sep 14 07:48:22.131601 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:22.131478    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rbgj4" event={"ID":"214873de-7ce7-4bc5-8817-4e82e603f5d2","Type":"ContainerStarted","Data":"4220a1520946746d641baf24331fa9f88f0ce37f4b91a3360cce18caf4084eae"}
Sep 14 07:48:22.140737 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:22.140510    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9" event={"ID":"e941a429-3862-413b-97fc-fceefa08d766","Type":"ContainerStarted","Data":"2ad284066f7d37f9b75a171adbff005c02ace743ca0caaf34253d0f76504517c"}
Sep 14 07:48:22.145478 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:22.145444    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tm584" event={"ID":"4079005f-d292-4b05-9d42-471d08e7d92a","Type":"ContainerStarted","Data":"ec93b8ed6d049c328ccacc2558f4182df20e7c7c4dd4d0f97f4504dbae226d39"}
Sep 14 07:48:22.151126 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:22.151092    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6xvpb" event={"ID":"25a3ecfc-dade-4e7c-8347-ce9f629d1f37","Type":"ContainerStarted","Data":"8d0fe8f86c66aff9d93c38fca54227d17f97f4405473fd3661fc0b7a6f861be9"}
Sep 14 07:48:22.164022 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:22.163987    2562 generic.go:356] "Generic (PLEG): container finished" podID="eb7e8b3289b4b5c21abcc1b439d2f0dd" containerID="f83cd63afcea1e4b32437b2b3ad857b5523b2c5f5db62704b7eee6a3281427c1" exitCode=0
Sep 14 07:48:22.164168 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:22.164083    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-28.ec2.internal" event={"ID":"eb7e8b3289b4b5c21abcc1b439d2f0dd","Type":"ContainerDied","Data":"f83cd63afcea1e4b32437b2b3ad857b5523b2c5f5db62704b7eee6a3281427c1"}
Sep 14 07:48:22.172444 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:22.172404    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mrgzf" event={"ID":"db75fca5-ca74-48e0-82f5-ee7514237f7f","Type":"ContainerStarted","Data":"987f97fd13cbd5dede59d50a53dd1869891bb9b1830058799073a5e9985a3cec"}
Sep 14 07:48:22.186769 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:22.186509    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h99nv" event={"ID":"f61681e5-1ce0-42ce-8ff4-9394378ede64","Type":"ContainerStarted","Data":"b1856322ba388e491383f37eb470d61d41948b914f496070bdf68124b817b929"}
Sep 14 07:48:22.193409 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:22.193379    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b2sjm" event={"ID":"d4c06442-0a48-45fb-a418-e51fa6a4cea4","Type":"ContainerStarted","Data":"71da0ff5c2f26f40aa4dcade6e239e78f3ae9857787098fd4968052193afe167"}
Sep 14 07:48:22.205085 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:22.205046    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-28.ec2.internal" event={"ID":"ef4817a2551205637f03cf112a16afcb","Type":"ContainerStarted","Data":"a44c00949a9bbf596c6f3ae555878ca4b5645e1b4c46ecc32c053bc5d0a9c1ba"}
Sep 14 07:48:22.216035 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:22.216003    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9" event={"ID":"59bed5f2-de61-4eae-b5a8-9bebc926467a","Type":"ContainerStarted","Data":"be824871f2d4abfb8ebd8b871f29978af0acd4b7fad67d60a9be50d9b27e8e7a"}
Sep 14 07:48:22.231230 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:22.231196    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pdfcv" event={"ID":"9594803d-c72d-4f5a-9b8d-e78d40e17d92","Type":"ContainerStarted","Data":"bb7cb1e0471e28744458402b8989d1a369e05c512f8a067c17e226d41c84798e"}
Sep 14 07:48:23.106883 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:23.106790    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:23.107474 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:23.106955    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mfj7" podUID="fd5d1c55-8829-4084-9b68-36cac615651b"
Sep 14 07:48:23.107474 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:23.107367    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:23.107474 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:23.107463    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbb69" podUID="4b4a6526-1631-4af9-9a20-cf80f53d3eac"
Sep 14 07:48:23.265769 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:23.265735    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-28.ec2.internal" event={"ID":"eb7e8b3289b4b5c21abcc1b439d2f0dd","Type":"ContainerStarted","Data":"51f9f9fd884b41a7041ed1471037d31f7a8921ae4e118f3939df2fe220e2896b"}
Sep 14 07:48:23.286639 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:23.286562    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-28.ec2.internal" podStartSLOduration=3.286537933 podStartE2EDuration="3.286537933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:22.250171654 +0000 UTC m=+4.706621652" watchObservedRunningTime="2026-09-14 07:48:23.286537933 +0000 UTC m=+5.742987933"
Sep 14 07:48:23.528302 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:23.528208    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret\") pod \"global-pull-secret-syncer-xbb69\" (UID: \"4b4a6526-1631-4af9-9a20-cf80f53d3eac\") " pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:23.528479 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:23.528303    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs\") pod \"network-metrics-daemon-4mfj7\" (UID: \"fd5d1c55-8829-4084-9b68-36cac615651b\") " pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:23.528479 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:23.528449    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:23.528708 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:23.528515    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs podName:fd5d1c55-8829-4084-9b68-36cac615651b nodeName:}" failed. No retries permitted until 2026-09-14 07:48:27.528494482 +0000 UTC m=+9.984944479 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs") pod "network-metrics-daemon-4mfj7" (UID: "fd5d1c55-8829-4084-9b68-36cac615651b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:23.528708 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:23.528607    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:23.528708 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:23.528648    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret podName:4b4a6526-1631-4af9-9a20-cf80f53d3eac nodeName:}" failed. No retries permitted until 2026-09-14 07:48:27.528634861 +0000 UTC m=+9.985084844 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret") pod "global-pull-secret-syncer-xbb69" (UID: "4b4a6526-1631-4af9-9a20-cf80f53d3eac") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:23.729588 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:23.729490    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77fds\" (UniqueName: \"kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds\") pod \"network-check-target-v7dql\" (UID: \"f3e50313-f82c-47b4-a66b-0bd7f9e227f4\") " pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:23.729785 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:23.729669    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:48:23.729785 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:23.729695    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:48:23.729785 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:23.729710    2562 projected.go:196] Error preparing data for projected volume kube-api-access-77fds for pod openshift-network-diagnostics/network-check-target-v7dql: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:23.729785 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:23.729770    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds podName:f3e50313-f82c-47b4-a66b-0bd7f9e227f4 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:27.729750487 +0000 UTC m=+10.186200464 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-77fds" (UniqueName: "kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds") pod "network-check-target-v7dql" (UID: "f3e50313-f82c-47b4-a66b-0bd7f9e227f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:24.107289 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:24.106759    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:24.107289 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:24.106904    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7dql" podUID="f3e50313-f82c-47b4-a66b-0bd7f9e227f4"
Sep 14 07:48:25.107195 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:25.107156    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:25.107378 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:25.107303    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbb69" podUID="4b4a6526-1631-4af9-9a20-cf80f53d3eac"
Sep 14 07:48:25.107723 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:25.107689    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:25.107912 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:25.107812    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mfj7" podUID="fd5d1c55-8829-4084-9b68-36cac615651b"
Sep 14 07:48:26.111368 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:26.111051    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:26.111368 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:26.111161    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7dql" podUID="f3e50313-f82c-47b4-a66b-0bd7f9e227f4"
Sep 14 07:48:27.107140 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:27.107052    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:27.107140 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:27.107084    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:27.107375 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:27.107206    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbb69" podUID="4b4a6526-1631-4af9-9a20-cf80f53d3eac"
Sep 14 07:48:27.107375 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:27.107339    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mfj7" podUID="fd5d1c55-8829-4084-9b68-36cac615651b"
Sep 14 07:48:27.562818 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:27.562694    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret\") pod \"global-pull-secret-syncer-xbb69\" (UID: \"4b4a6526-1631-4af9-9a20-cf80f53d3eac\") " pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:27.562818 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:27.562797    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs\") pod \"network-metrics-daemon-4mfj7\" (UID: \"fd5d1c55-8829-4084-9b68-36cac615651b\") " pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:27.563335 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:27.562934    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:27.563335 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:27.563006    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs podName:fd5d1c55-8829-4084-9b68-36cac615651b nodeName:}" failed. No retries permitted until 2026-09-14 07:48:35.562986801 +0000 UTC m=+18.019436788 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs") pod "network-metrics-daemon-4mfj7" (UID: "fd5d1c55-8829-4084-9b68-36cac615651b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:27.563456 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:27.563425    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:27.563514 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:27.563485    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret podName:4b4a6526-1631-4af9-9a20-cf80f53d3eac nodeName:}" failed. No retries permitted until 2026-09-14 07:48:35.563465829 +0000 UTC m=+18.019915803 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret") pod "global-pull-secret-syncer-xbb69" (UID: "4b4a6526-1631-4af9-9a20-cf80f53d3eac") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:27.764877 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:27.764838    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77fds\" (UniqueName: \"kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds\") pod \"network-check-target-v7dql\" (UID: \"f3e50313-f82c-47b4-a66b-0bd7f9e227f4\") " pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:27.765078 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:27.765049    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:48:27.765177 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:27.765086    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:48:27.765177 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:27.765102    2562 projected.go:196] Error preparing data for projected volume kube-api-access-77fds for pod openshift-network-diagnostics/network-check-target-v7dql: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:27.765177 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:27.765167    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds podName:f3e50313-f82c-47b4-a66b-0bd7f9e227f4 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:35.765146333 +0000 UTC m=+18.221596329 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-77fds" (UniqueName: "kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds") pod "network-check-target-v7dql" (UID: "f3e50313-f82c-47b4-a66b-0bd7f9e227f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:28.108481 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:28.108173    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:28.108700 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:28.108645    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7dql" podUID="f3e50313-f82c-47b4-a66b-0bd7f9e227f4"
Sep 14 07:48:29.106975 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:29.106939    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:29.107476 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:29.106995    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:29.107476 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:29.107082    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbb69" podUID="4b4a6526-1631-4af9-9a20-cf80f53d3eac"
Sep 14 07:48:29.107476 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:29.107406    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mfj7" podUID="fd5d1c55-8829-4084-9b68-36cac615651b"
Sep 14 07:48:30.107666 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:30.107334    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:30.107666 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:30.107483    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7dql" podUID="f3e50313-f82c-47b4-a66b-0bd7f9e227f4"
Sep 14 07:48:31.107305 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:31.107264    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:31.107484 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:31.107264    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:31.107484 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:31.107441    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mfj7" podUID="fd5d1c55-8829-4084-9b68-36cac615651b"
Sep 14 07:48:31.107632 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:31.107482    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbb69" podUID="4b4a6526-1631-4af9-9a20-cf80f53d3eac"
Sep 14 07:48:32.106516 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:32.106474    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:32.106939 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:32.106604    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7dql" podUID="f3e50313-f82c-47b4-a66b-0bd7f9e227f4"
Sep 14 07:48:33.106453 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:33.106408    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:33.106656 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:33.106464    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:33.106656 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:33.106549    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbb69" podUID="4b4a6526-1631-4af9-9a20-cf80f53d3eac"
Sep 14 07:48:33.107003 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:33.106695    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mfj7" podUID="fd5d1c55-8829-4084-9b68-36cac615651b"
Sep 14 07:48:34.106854 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:34.106818    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:34.107300 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:34.106950    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7dql" podUID="f3e50313-f82c-47b4-a66b-0bd7f9e227f4"
Sep 14 07:48:35.106456 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:35.106409    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:35.106696 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:35.106409    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:35.106696 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:35.106566    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbb69" podUID="4b4a6526-1631-4af9-9a20-cf80f53d3eac"
Sep 14 07:48:35.106696 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:35.106625    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mfj7" podUID="fd5d1c55-8829-4084-9b68-36cac615651b"
Sep 14 07:48:35.625485 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:35.625445    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs\") pod \"network-metrics-daemon-4mfj7\" (UID: \"fd5d1c55-8829-4084-9b68-36cac615651b\") " pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:35.625982 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:35.625525    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret\") pod \"global-pull-secret-syncer-xbb69\" (UID: \"4b4a6526-1631-4af9-9a20-cf80f53d3eac\") " pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:35.625982 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:35.625629    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:35.625982 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:35.625650    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:35.625982 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:35.625710    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs podName:fd5d1c55-8829-4084-9b68-36cac615651b nodeName:}" failed. No retries permitted until 2026-09-14 07:48:51.625687112 +0000 UTC m=+34.082137101 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs") pod "network-metrics-daemon-4mfj7" (UID: "fd5d1c55-8829-4084-9b68-36cac615651b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:35.625982 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:35.625733    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret podName:4b4a6526-1631-4af9-9a20-cf80f53d3eac nodeName:}" failed. No retries permitted until 2026-09-14 07:48:51.625722399 +0000 UTC m=+34.082172373 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret") pod "global-pull-secret-syncer-xbb69" (UID: "4b4a6526-1631-4af9-9a20-cf80f53d3eac") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:35.827075 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:35.827038    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77fds\" (UniqueName: \"kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds\") pod \"network-check-target-v7dql\" (UID: \"f3e50313-f82c-47b4-a66b-0bd7f9e227f4\") " pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:35.827258 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:35.827205    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:48:35.827258 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:35.827230    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:48:35.827258 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:35.827242    2562 projected.go:196] Error preparing data for projected volume kube-api-access-77fds for pod openshift-network-diagnostics/network-check-target-v7dql: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:35.827377 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:35.827300    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds podName:f3e50313-f82c-47b4-a66b-0bd7f9e227f4 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:51.82728252 +0000 UTC m=+34.283732506 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-77fds" (UniqueName: "kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds") pod "network-check-target-v7dql" (UID: "f3e50313-f82c-47b4-a66b-0bd7f9e227f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:36.106914 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:36.106834    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:36.107080 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:36.106961    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7dql" podUID="f3e50313-f82c-47b4-a66b-0bd7f9e227f4"
Sep 14 07:48:37.106456 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:37.106418    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:37.106904 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:37.106423    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:37.106904 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:37.106565    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbb69" podUID="4b4a6526-1631-4af9-9a20-cf80f53d3eac"
Sep 14 07:48:37.106904 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:37.106652    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mfj7" podUID="fd5d1c55-8829-4084-9b68-36cac615651b"
Sep 14 07:48:38.108062 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:38.108024    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:38.108529 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:38.108149    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7dql" podUID="f3e50313-f82c-47b4-a66b-0bd7f9e227f4"
Sep 14 07:48:39.106830 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:39.106793    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:39.107008 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:39.106793    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:39.107008 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:39.106911    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbb69" podUID="4b4a6526-1631-4af9-9a20-cf80f53d3eac"
Sep 14 07:48:39.107008 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:39.106987    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mfj7" podUID="fd5d1c55-8829-4084-9b68-36cac615651b"
Sep 14 07:48:40.106905 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.106720    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:40.107849 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:40.106980    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7dql" podUID="f3e50313-f82c-47b4-a66b-0bd7f9e227f4"
Sep 14 07:48:40.317171 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.317118    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pdfcv" event={"ID":"9594803d-c72d-4f5a-9b8d-e78d40e17d92","Type":"ContainerStarted","Data":"0d458b64f0eb1aeb2ddbb09999ca55eced26e2efb06f12b5e229fa18d71b6813"}
Sep 14 07:48:40.318869 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.318843    2562 generic.go:356] "Generic (PLEG): container finished" podID="214873de-7ce7-4bc5-8817-4e82e603f5d2" containerID="175e4d6a6cfddd3b860af5e14cc2595f3e8d2d7ad4232ac0a4ee9d7feef249ca" exitCode=0
Sep 14 07:48:40.319018 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.318918    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rbgj4" event={"ID":"214873de-7ce7-4bc5-8817-4e82e603f5d2","Type":"ContainerDied","Data":"175e4d6a6cfddd3b860af5e14cc2595f3e8d2d7ad4232ac0a4ee9d7feef249ca"}
Sep 14 07:48:40.322813 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.322740    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9" event={"ID":"e941a429-3862-413b-97fc-fceefa08d766","Type":"ContainerStarted","Data":"25fa26d5f5129dd5171882ba9dcc4a17b7289efca036fcae0f9151208891e1c1"}
Sep 14 07:48:40.322923 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.322833    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9" event={"ID":"e941a429-3862-413b-97fc-fceefa08d766","Type":"ContainerStarted","Data":"2f85654ca2744cc6cf65058d2ef64904aabfc89dacb1f45967d05d3b6c098665"}
Sep 14 07:48:40.322923 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.322872    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9" event={"ID":"e941a429-3862-413b-97fc-fceefa08d766","Type":"ContainerStarted","Data":"086e3d718f48ab295e83e4637b7e8c6e9cec3c49131e9cfc103408d2b0af7356"}
Sep 14 07:48:40.322923 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.322916    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9" event={"ID":"e941a429-3862-413b-97fc-fceefa08d766","Type":"ContainerStarted","Data":"94cbdf98a0e5ff09f36fe05e9d9dd6dadf76462753c215ad86f07544cb0da78e"}
Sep 14 07:48:40.323070 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.322957    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9" event={"ID":"e941a429-3862-413b-97fc-fceefa08d766","Type":"ContainerStarted","Data":"50e17c049399ab192d2d7e5265f984f736498a35f47fec69771f86dfb8302e5d"}
Sep 14 07:48:40.323070 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.322991    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9" event={"ID":"e941a429-3862-413b-97fc-fceefa08d766","Type":"ContainerStarted","Data":"0d739cdcdb716414441633db6f8f3241f455eebdac1540c860c5f73193ae9368"}
Sep 14 07:48:40.324410 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.324373    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tm584" event={"ID":"4079005f-d292-4b05-9d42-471d08e7d92a","Type":"ContainerStarted","Data":"3bbea12b7a5d173573990732701aeb59c01b3574d658dfe5b92539bb9c5acdcd"}
Sep 14 07:48:40.326084 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.326060    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6xvpb" event={"ID":"25a3ecfc-dade-4e7c-8347-ce9f629d1f37","Type":"ContainerStarted","Data":"110d958b6c8e0fe3f0208aa082d871e54a1b2c336143715680a56667d8b40b47"}
Sep 14 07:48:40.327653 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.327621    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h99nv" event={"ID":"f61681e5-1ce0-42ce-8ff4-9394378ede64","Type":"ContainerStarted","Data":"37854773df7eb59dfad04c623e0f2b8869d13cc123ef03526133d4ebd0259e26"}
Sep 14 07:48:40.329145 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.329124    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b2sjm" event={"ID":"d4c06442-0a48-45fb-a418-e51fa6a4cea4","Type":"ContainerStarted","Data":"00543b98a5029ac17c21c705cc08432eab28120b918a09584d5f35105bdd4933"}
Sep 14 07:48:40.330699 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.330679    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9" event={"ID":"59bed5f2-de61-4eae-b5a8-9bebc926467a","Type":"ContainerStarted","Data":"2e8224320a6c347704b91b5d8c5e968b62bcc8dc598f624906d9c6ed48fcaeab"}
Sep 14 07:48:40.332126 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.332084    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pdfcv" podStartSLOduration=3.286198601 podStartE2EDuration="21.332045897s" totalImagesPullingTime="18.045847296s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:40.332012402 +0000 UTC m=+22.788462398" watchObservedRunningTime="2026-09-14 07:48:40.332045897 +0000 UTC m=+22.788495892"
Sep 14 07:48:40.332271 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.332242    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-28.ec2.internal" podStartSLOduration=20.332230108 podStartE2EDuration="20.332230108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:23.287122194 +0000 UTC m=+5.743572191" watchObservedRunningTime="2026-09-14 07:48:40.332230108 +0000 UTC m=+22.788680104"
Sep 14 07:48:40.348625 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.348575    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tm584" podStartSLOduration=3.228911107 podStartE2EDuration="21.348559855s" totalImagesPullingTime="18.119648748s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:40.347770651 +0000 UTC m=+22.804220644" watchObservedRunningTime="2026-09-14 07:48:40.348559855 +0000 UTC m=+22.805009847"
Sep 14 07:48:40.362428 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.362381    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-b2sjm" podStartSLOduration=10.531626743 podStartE2EDuration="21.362366787s" totalImagesPullingTime="10.830740044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:40.36172542 +0000 UTC m=+22.818175414" watchObservedRunningTime="2026-09-14 07:48:40.362366787 +0000 UTC m=+22.818816797"
Sep 14 07:48:40.404049 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.404005    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-h99nv" podStartSLOduration=3.349775281 podStartE2EDuration="21.403990288s" totalImagesPullingTime="18.054215007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:40.403876241 +0000 UTC m=+22.860326234" watchObservedRunningTime="2026-09-14 07:48:40.403990288 +0000 UTC m=+22.860440280"
Sep 14 07:48:40.421196 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.421151    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-6xvpb" podStartSLOduration=3.365301569 podStartE2EDuration="21.421134813s" totalImagesPullingTime="18.055833244s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:40.420734074 +0000 UTC m=+22.877184068" watchObservedRunningTime="2026-09-14 07:48:40.421134813 +0000 UTC m=+22.877584805"
Sep 14 07:48:40.430493 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.430467    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:40.499706 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.499670    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:40.599951 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.599917    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:40.696704 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.696545    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 14 07:48:40.932893 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.932713    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-14T07:48:40.69669992Z","UUID":"3947a796-53fa-4e4a-bce2-1d8508659aab","Handler":null,"Name":"","Endpoint":""}
Sep 14 07:48:40.935561 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.935536    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 14 07:48:40.935707 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:40.935573    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 14 07:48:41.106583 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:41.106494    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:41.106583 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:41.106540    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:41.106810 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:41.106639    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbb69" podUID="4b4a6526-1631-4af9-9a20-cf80f53d3eac"
Sep 14 07:48:41.106865 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:41.106807    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mfj7" podUID="fd5d1c55-8829-4084-9b68-36cac615651b"
Sep 14 07:48:41.335588 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:41.335509    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9" event={"ID":"59bed5f2-de61-4eae-b5a8-9bebc926467a","Type":"ContainerStarted","Data":"609914ea558b6189621bb3b30bf692fd07e7d8c0ef4b69468bd52d9ef2032e40"}
Sep 14 07:48:41.336992 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:41.336970    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mrgzf" event={"ID":"db75fca5-ca74-48e0-82f5-ee7514237f7f","Type":"ContainerStarted","Data":"da653ecd1824d1180c580e0e4273e69487fdffa9520f560f9904174673678645"}
Sep 14 07:48:41.509018 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:41.508970    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:41.590333 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:41.590253    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:42.106898 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:42.106869    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:42.107056 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:42.106981    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7dql" podUID="f3e50313-f82c-47b4-a66b-0bd7f9e227f4"
Sep 14 07:48:42.342130 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:42.342035    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9" event={"ID":"59bed5f2-de61-4eae-b5a8-9bebc926467a","Type":"ContainerStarted","Data":"a7cd442aba87eabffdb355bd6f1186684a14f32cf3eb970adf64a2f8a11184a0"}
Sep 14 07:48:42.345995 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:42.345966    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9" event={"ID":"e941a429-3862-413b-97fc-fceefa08d766","Type":"ContainerStarted","Data":"a9ae9f56d9a12d60949f9108c7db9bcb74dfe45dde10989958fdc349888f04aa"}
Sep 14 07:48:42.362693 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:42.362639    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gr5x9" podStartSLOduration=3.096157431 podStartE2EDuration="23.362621208s" totalImagesPullingTime="20.266463777s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:42.362580527 +0000 UTC m=+24.819030524" watchObservedRunningTime="2026-09-14 07:48:42.362621208 +0000 UTC m=+24.819071202"
Sep 14 07:48:42.363314 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:42.363265    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mrgzf" podStartSLOduration=5.317272793 podStartE2EDuration="23.363256919s" totalImagesPullingTime="18.045984126s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:41.360511293 +0000 UTC m=+23.816961279" watchObservedRunningTime="2026-09-14 07:48:42.363256919 +0000 UTC m=+24.819706912"
Sep 14 07:48:43.106692 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:43.106648    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:43.106893 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:43.106653    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:43.106893 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:43.106798    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbb69" podUID="4b4a6526-1631-4af9-9a20-cf80f53d3eac"
Sep 14 07:48:43.106893 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:43.106873    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mfj7" podUID="fd5d1c55-8829-4084-9b68-36cac615651b"
Sep 14 07:48:43.901672 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:43.901632    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-b2sjm"
Sep 14 07:48:43.902227 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:43.902208    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-b2sjm"
Sep 14 07:48:44.107044 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:44.107016    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:44.107194 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:44.107143    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7dql" podUID="f3e50313-f82c-47b4-a66b-0bd7f9e227f4"
Sep 14 07:48:44.163967 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:44.163877    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-b2sjm"
Sep 14 07:48:44.164431 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:44.164408    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-b2sjm"
Sep 14 07:48:44.237414 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:44.237350    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:45.106681 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:45.106431    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:45.107319 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:45.106476    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:45.107319 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:45.106715    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbb69" podUID="4b4a6526-1631-4af9-9a20-cf80f53d3eac"
Sep 14 07:48:45.107319 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:45.106787    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mfj7" podUID="fd5d1c55-8829-4084-9b68-36cac615651b"
Sep 14 07:48:45.354235 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:45.354204    2562 generic.go:356] "Generic (PLEG): container finished" podID="214873de-7ce7-4bc5-8817-4e82e603f5d2" containerID="5d9cba4fd6e7360cd5ed9993d307f1f9cd5255485b763e97f12f0702f52f70f3" exitCode=0
Sep 14 07:48:45.354385 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:45.354272    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rbgj4" event={"ID":"214873de-7ce7-4bc5-8817-4e82e603f5d2","Type":"ContainerDied","Data":"5d9cba4fd6e7360cd5ed9993d307f1f9cd5255485b763e97f12f0702f52f70f3"}
Sep 14 07:48:45.358227 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:45.358205    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9" event={"ID":"e941a429-3862-413b-97fc-fceefa08d766","Type":"ContainerStarted","Data":"55e1ecba1f37c39b28f6a352c86170064ae6a16c5d4244bb570709f7b3d62726"}
Sep 14 07:48:45.410350 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:45.410294    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9" podStartSLOduration=8.306994531 podStartE2EDuration="26.410278043s" totalImagesPullingTime="18.103283512s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:45.409783681 +0000 UTC m=+27.866233687" watchObservedRunningTime="2026-09-14 07:48:45.410278043 +0000 UTC m=+27.866728035"
Sep 14 07:48:46.106862 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:46.106837    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:46.107377 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:46.106958    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7dql" podUID="f3e50313-f82c-47b4-a66b-0bd7f9e227f4"
Sep 14 07:48:46.338717 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:46.338435    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-v7dql"]
Sep 14 07:48:46.341472 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:46.341442    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xbb69"]
Sep 14 07:48:46.341662 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:46.341576    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:46.341732 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:46.341708    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbb69" podUID="4b4a6526-1631-4af9-9a20-cf80f53d3eac"
Sep 14 07:48:46.342116 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:46.342096    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4mfj7"]
Sep 14 07:48:46.342222 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:46.342177    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:46.342274 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:46.342257    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mfj7" podUID="fd5d1c55-8829-4084-9b68-36cac615651b"
Sep 14 07:48:46.363370 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:46.363341    2562 generic.go:356] "Generic (PLEG): container finished" podID="214873de-7ce7-4bc5-8817-4e82e603f5d2" containerID="66ef9ce3b75c4b17ac123e0aa928feee1833ebce70cea54d1e2a0b0d4dff15df" exitCode=0
Sep 14 07:48:46.363553 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:46.363380    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rbgj4" event={"ID":"214873de-7ce7-4bc5-8817-4e82e603f5d2","Type":"ContainerDied","Data":"66ef9ce3b75c4b17ac123e0aa928feee1833ebce70cea54d1e2a0b0d4dff15df"}
Sep 14 07:48:46.363553 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:46.363420    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:46.363553 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:46.363509    2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 14 07:48:46.363740 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:46.363598    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7dql" podUID="f3e50313-f82c-47b4-a66b-0bd7f9e227f4"
Sep 14 07:48:46.365323 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:46.364987    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:46.365323 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:46.365088    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:46.381672 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:46.381641    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:46.384759 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:46.384739    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:46.680808 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:46.680780    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:48:47.368623 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:47.368531    2562 generic.go:356] "Generic (PLEG): container finished" podID="214873de-7ce7-4bc5-8817-4e82e603f5d2" containerID="0ad3a792bdf063cbaa4a119f94f8b9691f753a6ed90a365610202fc48b4561e7" exitCode=0
Sep 14 07:48:47.368623 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:47.368611    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rbgj4" event={"ID":"214873de-7ce7-4bc5-8817-4e82e603f5d2","Type":"ContainerDied","Data":"0ad3a792bdf063cbaa4a119f94f8b9691f753a6ed90a365610202fc48b4561e7"}
Sep 14 07:48:48.108812 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:48.108436    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:48.108812 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:48.108727    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7dql" podUID="f3e50313-f82c-47b4-a66b-0bd7f9e227f4"
Sep 14 07:48:48.109034 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:48.108833    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:48.109034 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:48.108970    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mfj7" podUID="fd5d1c55-8829-4084-9b68-36cac615651b"
Sep 14 07:48:48.109034 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:48.109029    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:48.109171 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:48.109126    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbb69" podUID="4b4a6526-1631-4af9-9a20-cf80f53d3eac"
Sep 14 07:48:50.107339 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:50.107303    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:50.107904 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:50.107311    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:50.107904 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:50.107455    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:50.107904 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:50.107461    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mfj7" podUID="fd5d1c55-8829-4084-9b68-36cac615651b"
Sep 14 07:48:50.107904 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:50.107567    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbb69" podUID="4b4a6526-1631-4af9-9a20-cf80f53d3eac"
Sep 14 07:48:50.107904 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:50.107619    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7dql" podUID="f3e50313-f82c-47b4-a66b-0bd7f9e227f4"
Sep 14 07:48:50.229272 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:50.229216    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:50.325438 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:50.325397    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:51.641899 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:51.641860    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs\") pod \"network-metrics-daemon-4mfj7\" (UID: \"fd5d1c55-8829-4084-9b68-36cac615651b\") " pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:51.642386 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:51.641934    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret\") pod \"global-pull-secret-syncer-xbb69\" (UID: \"4b4a6526-1631-4af9-9a20-cf80f53d3eac\") " pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:51.642386 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:51.642032    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:51.642386 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:51.642078    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:51.642386 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:51.642107    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs podName:fd5d1c55-8829-4084-9b68-36cac615651b nodeName:}" failed. No retries permitted until 2026-09-14 07:49:23.64208836 +0000 UTC m=+66.098538334 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs") pod "network-metrics-daemon-4mfj7" (UID: "fd5d1c55-8829-4084-9b68-36cac615651b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:51.642386 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:51.642136    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret podName:4b4a6526-1631-4af9-9a20-cf80f53d3eac nodeName:}" failed. No retries permitted until 2026-09-14 07:49:23.642122349 +0000 UTC m=+66.098572321 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret") pod "global-pull-secret-syncer-xbb69" (UID: "4b4a6526-1631-4af9-9a20-cf80f53d3eac") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:51.843154 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:51.843122    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77fds\" (UniqueName: \"kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds\") pod \"network-check-target-v7dql\" (UID: \"f3e50313-f82c-47b4-a66b-0bd7f9e227f4\") " pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:51.843335 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:51.843252    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:48:51.843335 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:51.843267    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:48:51.843335 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:51.843275    2562 projected.go:196] Error preparing data for projected volume kube-api-access-77fds for pod openshift-network-diagnostics/network-check-target-v7dql: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:51.843335 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:51.843323    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds podName:f3e50313-f82c-47b4-a66b-0bd7f9e227f4 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:23.843310083 +0000 UTC m=+66.299760070 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-77fds" (UniqueName: "kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds") pod "network-check-target-v7dql" (UID: "f3e50313-f82c-47b4-a66b-0bd7f9e227f4") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:52.106667 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.106637    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:52.106667 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.106668    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:52.106862 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.106642    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:52.107211 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:52.107175    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xbb69" podUID="4b4a6526-1631-4af9-9a20-cf80f53d3eac"
Sep 14 07:48:52.110413 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:52.107745    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-v7dql" podUID="f3e50313-f82c-47b4-a66b-0bd7f9e227f4"
Sep 14 07:48:52.110413 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:52.107899    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4mfj7" podUID="fd5d1c55-8829-4084-9b68-36cac615651b"
Sep 14 07:48:52.270145 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.269934    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-28.ec2.internal" event="NodeReady"
Sep 14 07:48:52.270265 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.270218    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 14 07:48:52.374356 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.374322    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mfdkf"]
Sep 14 07:48:52.382580 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.382554    2562 generic.go:356] "Generic (PLEG): container finished" podID="214873de-7ce7-4bc5-8817-4e82e603f5d2" containerID="2526ec13876842a2602d04e48995542041dd02438e18adf1bf8c0febc5ea42cf" exitCode=0
Sep 14 07:48:52.387008 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.386993    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5kvk4"]
Sep 14 07:48:52.387112 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.387096    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mfdkf"
Sep 14 07:48:52.389463 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.389443    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 14 07:48:52.390381 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.390362    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7986b\""
Sep 14 07:48:52.390479 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.390404    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 14 07:48:52.405160 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.405138    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5kvk4"]
Sep 14 07:48:52.405263 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.405167    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rbgj4" event={"ID":"214873de-7ce7-4bc5-8817-4e82e603f5d2","Type":"ContainerDied","Data":"2526ec13876842a2602d04e48995542041dd02438e18adf1bf8c0febc5ea42cf"}
Sep 14 07:48:52.405263 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.405248    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5kvk4"
Sep 14 07:48:52.411410 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.411387    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 14 07:48:52.411603 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.411419    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 14 07:48:52.411603 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.411461    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 14 07:48:52.411603 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.411552    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wv9fj\""
Sep 14 07:48:52.418814 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.418790    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mfdkf"]
Sep 14 07:48:52.447832 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.447800    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/649d1d31-0095-4b78-89e6-cecf97d8d72a-config-volume\") pod \"dns-default-mfdkf\" (UID: \"649d1d31-0095-4b78-89e6-cecf97d8d72a\") " pod="openshift-dns/dns-default-mfdkf"
Sep 14 07:48:52.447832 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.447842    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/649d1d31-0095-4b78-89e6-cecf97d8d72a-tmp-dir\") pod \"dns-default-mfdkf\" (UID: \"649d1d31-0095-4b78-89e6-cecf97d8d72a\") " pod="openshift-dns/dns-default-mfdkf"
Sep 14 07:48:52.448032 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.447947    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfx2s\" (UniqueName: \"kubernetes.io/projected/649d1d31-0095-4b78-89e6-cecf97d8d72a-kube-api-access-rfx2s\") pod \"dns-default-mfdkf\" (UID: \"649d1d31-0095-4b78-89e6-cecf97d8d72a\") " pod="openshift-dns/dns-default-mfdkf"
Sep 14 07:48:52.448032 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.447996    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/649d1d31-0095-4b78-89e6-cecf97d8d72a-metrics-tls\") pod \"dns-default-mfdkf\" (UID: \"649d1d31-0095-4b78-89e6-cecf97d8d72a\") " pod="openshift-dns/dns-default-mfdkf"
Sep 14 07:48:52.548913 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.548884    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7cgj\" (UniqueName: \"kubernetes.io/projected/ad3788b1-dd2a-4804-bfb8-3ac6b77e03e1-kube-api-access-m7cgj\") pod \"ingress-canary-5kvk4\" (UID: \"ad3788b1-dd2a-4804-bfb8-3ac6b77e03e1\") " pod="openshift-ingress-canary/ingress-canary-5kvk4"
Sep 14 07:48:52.549032 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.548926    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/649d1d31-0095-4b78-89e6-cecf97d8d72a-config-volume\") pod \"dns-default-mfdkf\" (UID: \"649d1d31-0095-4b78-89e6-cecf97d8d72a\") " pod="openshift-dns/dns-default-mfdkf"
Sep 14 07:48:52.549032 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.548950    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/649d1d31-0095-4b78-89e6-cecf97d8d72a-tmp-dir\") pod \"dns-default-mfdkf\" (UID: \"649d1d31-0095-4b78-89e6-cecf97d8d72a\") " pod="openshift-dns/dns-default-mfdkf"
Sep 14 07:48:52.549032 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.548982    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad3788b1-dd2a-4804-bfb8-3ac6b77e03e1-cert\") pod \"ingress-canary-5kvk4\" (UID: \"ad3788b1-dd2a-4804-bfb8-3ac6b77e03e1\") " pod="openshift-ingress-canary/ingress-canary-5kvk4"
Sep 14 07:48:52.549145 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.549043    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfx2s\" (UniqueName: \"kubernetes.io/projected/649d1d31-0095-4b78-89e6-cecf97d8d72a-kube-api-access-rfx2s\") pod \"dns-default-mfdkf\" (UID: \"649d1d31-0095-4b78-89e6-cecf97d8d72a\") " pod="openshift-dns/dns-default-mfdkf"
Sep 14 07:48:52.549145 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.549109    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/649d1d31-0095-4b78-89e6-cecf97d8d72a-metrics-tls\") pod \"dns-default-mfdkf\" (UID: \"649d1d31-0095-4b78-89e6-cecf97d8d72a\") " pod="openshift-dns/dns-default-mfdkf"
Sep 14 07:48:52.549243 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:52.549219    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:48:52.549302 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.549248    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/649d1d31-0095-4b78-89e6-cecf97d8d72a-tmp-dir\") pod \"dns-default-mfdkf\" (UID: \"649d1d31-0095-4b78-89e6-cecf97d8d72a\") " pod="openshift-dns/dns-default-mfdkf"
Sep 14 07:48:52.549302 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:52.549293    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/649d1d31-0095-4b78-89e6-cecf97d8d72a-metrics-tls podName:649d1d31-0095-4b78-89e6-cecf97d8d72a nodeName:}" failed. No retries permitted until 2026-09-14 07:48:53.049271678 +0000 UTC m=+35.505721660 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/649d1d31-0095-4b78-89e6-cecf97d8d72a-metrics-tls") pod "dns-default-mfdkf" (UID: "649d1d31-0095-4b78-89e6-cecf97d8d72a") : secret "dns-default-metrics-tls" not found
Sep 14 07:48:52.549451 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.549433    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/649d1d31-0095-4b78-89e6-cecf97d8d72a-config-volume\") pod \"dns-default-mfdkf\" (UID: \"649d1d31-0095-4b78-89e6-cecf97d8d72a\") " pod="openshift-dns/dns-default-mfdkf"
Sep 14 07:48:52.551383 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.551359    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:52.563675 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.563643    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfx2s\" (UniqueName: \"kubernetes.io/projected/649d1d31-0095-4b78-89e6-cecf97d8d72a-kube-api-access-rfx2s\") pod \"dns-default-mfdkf\" (UID: \"649d1d31-0095-4b78-89e6-cecf97d8d72a\") " pod="openshift-dns/dns-default-mfdkf"
Sep 14 07:48:52.649409 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.649382    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7cgj\" (UniqueName: \"kubernetes.io/projected/ad3788b1-dd2a-4804-bfb8-3ac6b77e03e1-kube-api-access-m7cgj\") pod \"ingress-canary-5kvk4\" (UID: \"ad3788b1-dd2a-4804-bfb8-3ac6b77e03e1\") " pod="openshift-ingress-canary/ingress-canary-5kvk4"
Sep 14 07:48:52.649886 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.649419    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad3788b1-dd2a-4804-bfb8-3ac6b77e03e1-cert\") pod \"ingress-canary-5kvk4\" (UID: \"ad3788b1-dd2a-4804-bfb8-3ac6b77e03e1\") " pod="openshift-ingress-canary/ingress-canary-5kvk4"
Sep 14 07:48:52.649886 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.649428    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:52.651874 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.651853    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad3788b1-dd2a-4804-bfb8-3ac6b77e03e1-cert\") pod \"ingress-canary-5kvk4\" (UID: \"ad3788b1-dd2a-4804-bfb8-3ac6b77e03e1\") " pod="openshift-ingress-canary/ingress-canary-5kvk4"
Sep 14 07:48:52.658814 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.658795    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7cgj\" (UniqueName: \"kubernetes.io/projected/ad3788b1-dd2a-4804-bfb8-3ac6b77e03e1-kube-api-access-m7cgj\") pod \"ingress-canary-5kvk4\" (UID: \"ad3788b1-dd2a-4804-bfb8-3ac6b77e03e1\") " pod="openshift-ingress-canary/ingress-canary-5kvk4"
Sep 14 07:48:52.713875 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:52.713844    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5kvk4"
Sep 14 07:48:53.035172 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:53.035142    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5kvk4"]
Sep 14 07:48:53.038746 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:53.038720    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad3788b1_dd2a_4804_bfb8_3ac6b77e03e1.slice/crio-20e4e3c60e6c04d4f394ce9d8d6d481e208ad6016b75b357b814bcc30f0de320 WatchSource:0}: Error finding container 20e4e3c60e6c04d4f394ce9d8d6d481e208ad6016b75b357b814bcc30f0de320: Status 404 returned error can't find the container with id 20e4e3c60e6c04d4f394ce9d8d6d481e208ad6016b75b357b814bcc30f0de320
Sep 14 07:48:53.052360 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:53.052340    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/649d1d31-0095-4b78-89e6-cecf97d8d72a-metrics-tls\") pod \"dns-default-mfdkf\" (UID: \"649d1d31-0095-4b78-89e6-cecf97d8d72a\") " pod="openshift-dns/dns-default-mfdkf"
Sep 14 07:48:53.052456 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:53.052443    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:48:53.052521 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:53.052511    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/649d1d31-0095-4b78-89e6-cecf97d8d72a-metrics-tls podName:649d1d31-0095-4b78-89e6-cecf97d8d72a nodeName:}" failed. No retries permitted until 2026-09-14 07:48:54.052481592 +0000 UTC m=+36.508931565 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/649d1d31-0095-4b78-89e6-cecf97d8d72a-metrics-tls") pod "dns-default-mfdkf" (UID: "649d1d31-0095-4b78-89e6-cecf97d8d72a") : secret "dns-default-metrics-tls" not found
Sep 14 07:48:53.388344 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:53.388310    2562 generic.go:356] "Generic (PLEG): container finished" podID="214873de-7ce7-4bc5-8817-4e82e603f5d2" containerID="fce1552588466b601652d9cb2acf4d1bae61bc5664d449e36766bef2b9d0e288" exitCode=0
Sep 14 07:48:53.388530 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:53.388346    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rbgj4" event={"ID":"214873de-7ce7-4bc5-8817-4e82e603f5d2","Type":"ContainerDied","Data":"fce1552588466b601652d9cb2acf4d1bae61bc5664d449e36766bef2b9d0e288"}
Sep 14 07:48:53.389592 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:53.389571    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5kvk4" event={"ID":"ad3788b1-dd2a-4804-bfb8-3ac6b77e03e1","Type":"ContainerStarted","Data":"20e4e3c60e6c04d4f394ce9d8d6d481e208ad6016b75b357b814bcc30f0de320"}
Sep 14 07:48:54.060222 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.059961    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/649d1d31-0095-4b78-89e6-cecf97d8d72a-metrics-tls\") pod \"dns-default-mfdkf\" (UID: \"649d1d31-0095-4b78-89e6-cecf97d8d72a\") " pod="openshift-dns/dns-default-mfdkf"
Sep 14 07:48:54.060976 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:54.060126    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:48:54.060976 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:54.060370    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/649d1d31-0095-4b78-89e6-cecf97d8d72a-metrics-tls podName:649d1d31-0095-4b78-89e6-cecf97d8d72a nodeName:}" failed. No retries permitted until 2026-09-14 07:48:56.060346867 +0000 UTC m=+38.516796854 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/649d1d31-0095-4b78-89e6-cecf97d8d72a-metrics-tls") pod "dns-default-mfdkf" (UID: "649d1d31-0095-4b78-89e6-cecf97d8d72a") : secret "dns-default-metrics-tls" not found
Sep 14 07:48:54.107254 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.107223    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:48:54.107432 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.107223    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:48:54.107759 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.107223    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:48:54.114393 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.114367    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 07:48:54.114908 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.114885    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 07:48:54.116932 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.116742    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 07:48:54.116932 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.116824    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bd4sd\""
Sep 14 07:48:54.116932 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.116904    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 07:48:54.121094 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.121076    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qxrn8\""
Sep 14 07:48:54.396873 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.396787    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rbgj4" event={"ID":"214873de-7ce7-4bc5-8817-4e82e603f5d2","Type":"ContainerStarted","Data":"1d7ade1f559381ff3578204610cd03f7e2b80357b12fa071a8f0d5b3df2e748f"}
Sep 14 07:48:54.403205 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.403174    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:54.423346 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.423295    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rbgj4" podStartSLOduration=7.250153544 podStartE2EDuration="35.423277534s" totalImagesPullingTime="28.17312399s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:54.422985588 +0000 UTC m=+36.879435599" watchObservedRunningTime="2026-09-14 07:48:54.423277534 +0000 UTC m=+36.879727526"
Sep 14 07:48:54.553894 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.553857    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:54.627077 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.627038    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:54.679951 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.679841    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-z8xz5"]
Sep 14 07:48:54.691303 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.691268    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-z8xz5"]
Sep 14 07:48:54.691449 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.691364    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-z8xz5"
Sep 14 07:48:54.693672 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.693636    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 14 07:48:54.693816 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.693692    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-qzf9w\""
Sep 14 07:48:54.693816 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.693713    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 14 07:48:54.717186 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.717154    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:54.765802 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.765768    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t24z\" (UniqueName: \"kubernetes.io/projected/f2032204-d7ba-45f2-81d3-69271762729b-kube-api-access-6t24z\") pod \"migrator-8b4d697c5-z8xz5\" (UID: \"f2032204-d7ba-45f2-81d3-69271762729b\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-z8xz5"
Sep 14 07:48:54.866965 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.866838    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t24z\" (UniqueName: \"kubernetes.io/projected/f2032204-d7ba-45f2-81d3-69271762729b-kube-api-access-6t24z\") pod \"migrator-8b4d697c5-z8xz5\" (UID: \"f2032204-d7ba-45f2-81d3-69271762729b\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-z8xz5"
Sep 14 07:48:54.883913 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.883881    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:54.884072 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.883949    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t24z\" (UniqueName: \"kubernetes.io/projected/f2032204-d7ba-45f2-81d3-69271762729b-kube-api-access-6t24z\") pod \"migrator-8b4d697c5-z8xz5\" (UID: \"f2032204-d7ba-45f2-81d3-69271762729b\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-z8xz5"
Sep 14 07:48:54.944376 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:54.944291    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:55.000657 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.000620    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-z8xz5"
Sep 14 07:48:55.025361 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.025332    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9xx2k"]
Sep 14 07:48:55.031466 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.031431    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:48:55.034816 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.034789    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:55.034972 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.034956    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 14 07:48:55.035095 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.035075    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 14 07:48:55.035584 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.035564    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 14 07:48:55.035698 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.035633    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 14 07:48:55.035698 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.035676    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k6zzx\""
Sep 14 07:48:55.042004 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.041983    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9xx2k"]
Sep 14 07:48:55.129587 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.129484    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:55.169563 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.169528    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6c2d43af-8e1f-4a7a-a94d-195c15988364-crio-socket\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:48:55.169563 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.169576    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c2d43af-8e1f-4a7a-a94d-195c15988364-data-volume\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:48:55.169792 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.169600    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c2d43af-8e1f-4a7a-a94d-195c15988364-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:48:55.169792 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.169687    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2d43af-8e1f-4a7a-a94d-195c15988364-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:48:55.169792 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.169726    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnqlq\" (UniqueName: \"kubernetes.io/projected/6c2d43af-8e1f-4a7a-a94d-195c15988364-kube-api-access-pnqlq\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:48:55.232457 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.232333    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:55.270246 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.270207    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6c2d43af-8e1f-4a7a-a94d-195c15988364-crio-socket\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:48:55.270413 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.270263    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c2d43af-8e1f-4a7a-a94d-195c15988364-data-volume\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:48:55.270476 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.270405    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c2d43af-8e1f-4a7a-a94d-195c15988364-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:48:55.270476 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.270464    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2d43af-8e1f-4a7a-a94d-195c15988364-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:48:55.270612 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.270493    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnqlq\" (UniqueName: \"kubernetes.io/projected/6c2d43af-8e1f-4a7a-a94d-195c15988364-kube-api-access-pnqlq\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:48:55.270612 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.270535    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6c2d43af-8e1f-4a7a-a94d-195c15988364-crio-socket\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:48:55.270612 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:55.270543    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:48:55.270612 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:55.270603    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:55.270848 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.270641    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c2d43af-8e1f-4a7a-a94d-195c15988364-data-volume\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:48:55.270848 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:55.270651    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c2d43af-8e1f-4a7a-a94d-195c15988364-kube-rbac-proxy-cm podName:6c2d43af-8e1f-4a7a-a94d-195c15988364 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:55.770628872 +0000 UTC m=+38.227078894 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c2d43af-8e1f-4a7a-a94d-195c15988364-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9xx2k" (UID: "6c2d43af-8e1f-4a7a-a94d-195c15988364") : configmap "kube-rbac-proxy" not found
Sep 14 07:48:55.270848 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:55.270677    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c2d43af-8e1f-4a7a-a94d-195c15988364-insights-runtime-extractor-tls podName:6c2d43af-8e1f-4a7a-a94d-195c15988364 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:55.770658941 +0000 UTC m=+38.227108925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c2d43af-8e1f-4a7a-a94d-195c15988364-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9xx2k" (UID: "6c2d43af-8e1f-4a7a-a94d-195c15988364") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:55.280965 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.280935    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnqlq\" (UniqueName: \"kubernetes.io/projected/6c2d43af-8e1f-4a7a-a94d-195c15988364-kube-api-access-pnqlq\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:48:55.775028 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.774996    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c2d43af-8e1f-4a7a-a94d-195c15988364-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:48:55.775028 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.775032    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2d43af-8e1f-4a7a-a94d-195c15988364-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:48:55.775219 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:55.775120    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:48:55.775219 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:55.775203    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c2d43af-8e1f-4a7a-a94d-195c15988364-kube-rbac-proxy-cm podName:6c2d43af-8e1f-4a7a-a94d-195c15988364 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:56.775186573 +0000 UTC m=+39.231636543 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c2d43af-8e1f-4a7a-a94d-195c15988364-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9xx2k" (UID: "6c2d43af-8e1f-4a7a-a94d-195c15988364") : configmap "kube-rbac-proxy" not found
Sep 14 07:48:55.775219 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:55.775204    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:55.775334 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:55.775254    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c2d43af-8e1f-4a7a-a94d-195c15988364-insights-runtime-extractor-tls podName:6c2d43af-8e1f-4a7a-a94d-195c15988364 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:56.775241834 +0000 UTC m=+39.231691810 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c2d43af-8e1f-4a7a-a94d-195c15988364-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9xx2k" (UID: "6c2d43af-8e1f-4a7a-a94d-195c15988364") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:55.965688 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:55.965658    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-z8xz5"]
Sep 14 07:48:55.969698 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:55.969666    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2032204_d7ba_45f2_81d3_69271762729b.slice/crio-13fee75d1ebfb7d5a46fde846b0b9e445db20ae2e83268d62d11385b4da123c0 WatchSource:0}: Error finding container 13fee75d1ebfb7d5a46fde846b0b9e445db20ae2e83268d62d11385b4da123c0: Status 404 returned error can't find the container with id 13fee75d1ebfb7d5a46fde846b0b9e445db20ae2e83268d62d11385b4da123c0
Sep 14 07:48:56.077965 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.077540    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/649d1d31-0095-4b78-89e6-cecf97d8d72a-metrics-tls\") pod \"dns-default-mfdkf\" (UID: \"649d1d31-0095-4b78-89e6-cecf97d8d72a\") " pod="openshift-dns/dns-default-mfdkf"
Sep 14 07:48:56.077965 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:56.077771    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:48:56.077965 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:56.077903    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/649d1d31-0095-4b78-89e6-cecf97d8d72a-metrics-tls podName:649d1d31-0095-4b78-89e6-cecf97d8d72a nodeName:}" failed. No retries permitted until 2026-09-14 07:49:00.07788273 +0000 UTC m=+42.534332713 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/649d1d31-0095-4b78-89e6-cecf97d8d72a-metrics-tls") pod "dns-default-mfdkf" (UID: "649d1d31-0095-4b78-89e6-cecf97d8d72a") : secret "dns-default-metrics-tls" not found
Sep 14 07:48:56.125848 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.125811    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:56.189002 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.188966    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:56.402837 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.402755    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5kvk4" event={"ID":"ad3788b1-dd2a-4804-bfb8-3ac6b77e03e1","Type":"ContainerStarted","Data":"8548939a5b9a06136c291be2ee143bf6a7a58c27fe88d192438c65fed97d31a7"}
Sep 14 07:48:56.403960 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.403936    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-z8xz5" event={"ID":"f2032204-d7ba-45f2-81d3-69271762729b","Type":"ContainerStarted","Data":"13fee75d1ebfb7d5a46fde846b0b9e445db20ae2e83268d62d11385b4da123c0"}
Sep 14 07:48:56.726633 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.726531    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5kvk4" podStartSLOduration=1.8133548670000001 podStartE2EDuration="4.726492647s" totalImagesPullingTime="2.91313778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:56.422941735 +0000 UTC m=+38.879391727" watchObservedRunningTime="2026-09-14 07:48:56.726492647 +0000 UTC m=+39.182942638"
Sep 14 07:48:56.726848 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.726832    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-ltmm5"]
Sep 14 07:48:56.750787 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.750747    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-ltmm5"]
Sep 14 07:48:56.750958 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.750898    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-ltmm5"
Sep 14 07:48:56.753265 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.753237    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-57zrq\""
Sep 14 07:48:56.753410 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.753288    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 14 07:48:56.753410 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.753237    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 14 07:48:56.753666 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.753649    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 14 07:48:56.753774 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.753673    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 14 07:48:56.783686 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.783659    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c2d43af-8e1f-4a7a-a94d-195c15988364-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:48:56.783849 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.783696    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2d43af-8e1f-4a7a-a94d-195c15988364-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:48:56.783849 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:56.783789    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:48:56.783938 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:56.783864    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c2d43af-8e1f-4a7a-a94d-195c15988364-kube-rbac-proxy-cm podName:6c2d43af-8e1f-4a7a-a94d-195c15988364 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:58.783843795 +0000 UTC m=+41.240293784 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c2d43af-8e1f-4a7a-a94d-195c15988364-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9xx2k" (UID: "6c2d43af-8e1f-4a7a-a94d-195c15988364") : configmap "kube-rbac-proxy" not found
Sep 14 07:48:56.783938 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:56.783868    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:56.783938 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:56.783923    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c2d43af-8e1f-4a7a-a94d-195c15988364-insights-runtime-extractor-tls podName:6c2d43af-8e1f-4a7a-a94d-195c15988364 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:58.7839073 +0000 UTC m=+41.240357285 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c2d43af-8e1f-4a7a-a94d-195c15988364-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9xx2k" (UID: "6c2d43af-8e1f-4a7a-a94d-195c15988364") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:56.832628 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.832587    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:56.884462 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.884433    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c58rg\" (UniqueName: \"kubernetes.io/projected/b73a3fe9-053a-41b9-b85b-93dace0cf9a2-kube-api-access-c58rg\") pod \"service-ca-568869fdc5-ltmm5\" (UID: \"b73a3fe9-053a-41b9-b85b-93dace0cf9a2\") " pod="openshift-service-ca/service-ca-568869fdc5-ltmm5"
Sep 14 07:48:56.884656 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.884472    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b73a3fe9-053a-41b9-b85b-93dace0cf9a2-signing-cabundle\") pod \"service-ca-568869fdc5-ltmm5\" (UID: \"b73a3fe9-053a-41b9-b85b-93dace0cf9a2\") " pod="openshift-service-ca/service-ca-568869fdc5-ltmm5"
Sep 14 07:48:56.884656 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.884516    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b73a3fe9-053a-41b9-b85b-93dace0cf9a2-signing-key\") pod \"service-ca-568869fdc5-ltmm5\" (UID: \"b73a3fe9-053a-41b9-b85b-93dace0cf9a2\") " pod="openshift-service-ca/service-ca-568869fdc5-ltmm5"
Sep 14 07:48:56.938092 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.938053    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:56.985593 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.985519    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c58rg\" (UniqueName: \"kubernetes.io/projected/b73a3fe9-053a-41b9-b85b-93dace0cf9a2-kube-api-access-c58rg\") pod \"service-ca-568869fdc5-ltmm5\" (UID: \"b73a3fe9-053a-41b9-b85b-93dace0cf9a2\") " pod="openshift-service-ca/service-ca-568869fdc5-ltmm5"
Sep 14 07:48:56.985593 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.985577    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b73a3fe9-053a-41b9-b85b-93dace0cf9a2-signing-cabundle\") pod \"service-ca-568869fdc5-ltmm5\" (UID: \"b73a3fe9-053a-41b9-b85b-93dace0cf9a2\") " pod="openshift-service-ca/service-ca-568869fdc5-ltmm5"
Sep 14 07:48:56.985789 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.985614    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b73a3fe9-053a-41b9-b85b-93dace0cf9a2-signing-key\") pod \"service-ca-568869fdc5-ltmm5\" (UID: \"b73a3fe9-053a-41b9-b85b-93dace0cf9a2\") " pod="openshift-service-ca/service-ca-568869fdc5-ltmm5"
Sep 14 07:48:56.986293 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.986271    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b73a3fe9-053a-41b9-b85b-93dace0cf9a2-signing-cabundle\") pod \"service-ca-568869fdc5-ltmm5\" (UID: \"b73a3fe9-053a-41b9-b85b-93dace0cf9a2\") " pod="openshift-service-ca/service-ca-568869fdc5-ltmm5"
Sep 14 07:48:56.989768 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.989743    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b73a3fe9-053a-41b9-b85b-93dace0cf9a2-signing-key\") pod \"service-ca-568869fdc5-ltmm5\" (UID: \"b73a3fe9-053a-41b9-b85b-93dace0cf9a2\") " pod="openshift-service-ca/service-ca-568869fdc5-ltmm5"
Sep 14 07:48:56.996000 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:56.995978    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c58rg\" (UniqueName: \"kubernetes.io/projected/b73a3fe9-053a-41b9-b85b-93dace0cf9a2-kube-api-access-c58rg\") pod \"service-ca-568869fdc5-ltmm5\" (UID: \"b73a3fe9-053a-41b9-b85b-93dace0cf9a2\") " pod="openshift-service-ca/service-ca-568869fdc5-ltmm5"
Sep 14 07:48:57.062222 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:57.062187    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-ltmm5"
Sep 14 07:48:57.253348 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:57.253275    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:57.308195 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:57.308159    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:57.456448 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:57.456421    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-ltmm5"]
Sep 14 07:48:57.460094 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:48:57.460070    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb73a3fe9_053a_41b9_b85b_93dace0cf9a2.slice/crio-03e4da180ef81073acbcc07597eaacb860dcd4bdba7c4dbf8fb82c97e51e3284 WatchSource:0}: Error finding container 03e4da180ef81073acbcc07597eaacb860dcd4bdba7c4dbf8fb82c97e51e3284: Status 404 returned error can't find the container with id 03e4da180ef81073acbcc07597eaacb860dcd4bdba7c4dbf8fb82c97e51e3284
Sep 14 07:48:57.571940 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:57.571919    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6xvpb_25a3ecfc-dade-4e7c-8347-ce9f629d1f37/dns-node-resolver/0.log"
Sep 14 07:48:57.832953 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:57.832867    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:48:58.410715 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:58.410674    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-z8xz5" event={"ID":"f2032204-d7ba-45f2-81d3-69271762729b","Type":"ContainerStarted","Data":"f4a55f9508e4e5f5d06e707b2342d8a31783710a9efdddb7bb7ee20d6d4647f3"}
Sep 14 07:48:58.411174 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:58.410741    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-z8xz5" event={"ID":"f2032204-d7ba-45f2-81d3-69271762729b","Type":"ContainerStarted","Data":"5d50943900e61f5a118142c88b6b5d00615d92ec62bef013d1117cdc1d072efa"}
Sep 14 07:48:58.412004 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:58.411977    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-ltmm5" event={"ID":"b73a3fe9-053a-41b9-b85b-93dace0cf9a2","Type":"ContainerStarted","Data":"03e4da180ef81073acbcc07597eaacb860dcd4bdba7c4dbf8fb82c97e51e3284"}
Sep 14 07:48:58.434459 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:58.434400    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-z8xz5" podStartSLOduration=3.070780647 podStartE2EDuration="4.43438129s" totalImagesPullingTime="1.363600643s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:58.433872617 +0000 UTC m=+40.890322611" watchObservedRunningTime="2026-09-14 07:48:58.43438129 +0000 UTC m=+40.890831284"
Sep 14 07:48:58.772675 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:58.772594    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pdfcv_9594803d-c72d-4f5a-9b8d-e78d40e17d92/node-ca/0.log"
Sep 14 07:48:58.799923 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:58.799881    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c2d43af-8e1f-4a7a-a94d-195c15988364-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:48:58.799923 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:58.799924    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2d43af-8e1f-4a7a-a94d-195c15988364-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:48:58.800115 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:58.800012    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:48:58.800115 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:58.800022    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:58.800115 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:58.800073    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c2d43af-8e1f-4a7a-a94d-195c15988364-insights-runtime-extractor-tls podName:6c2d43af-8e1f-4a7a-a94d-195c15988364 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:02.800060149 +0000 UTC m=+45.256510121 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c2d43af-8e1f-4a7a-a94d-195c15988364-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9xx2k" (UID: "6c2d43af-8e1f-4a7a-a94d-195c15988364") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:58.800115 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:48:58.800087    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c2d43af-8e1f-4a7a-a94d-195c15988364-kube-rbac-proxy-cm podName:6c2d43af-8e1f-4a7a-a94d-195c15988364 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:02.800080187 +0000 UTC m=+45.256530157 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c2d43af-8e1f-4a7a-a94d-195c15988364-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9xx2k" (UID: "6c2d43af-8e1f-4a7a-a94d-195c15988364") : configmap "kube-rbac-proxy" not found
Sep 14 07:48:59.374956 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:59.374916    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5kvk4_ad3788b1-dd2a-4804-bfb8-3ac6b77e03e1/serve-healthcheck-canary/0.log"
Sep 14 07:48:59.973293 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:48:59.973267    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-z8xz5_f2032204-d7ba-45f2-81d3-69271762729b/migrator/0.log"
Sep 14 07:49:00.110362 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:00.110280    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/649d1d31-0095-4b78-89e6-cecf97d8d72a-metrics-tls\") pod \"dns-default-mfdkf\" (UID: \"649d1d31-0095-4b78-89e6-cecf97d8d72a\") " pod="openshift-dns/dns-default-mfdkf"
Sep 14 07:49:00.110530 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:49:00.110430    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:49:00.110530 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:49:00.110487    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/649d1d31-0095-4b78-89e6-cecf97d8d72a-metrics-tls podName:649d1d31-0095-4b78-89e6-cecf97d8d72a nodeName:}" failed. No retries permitted until 2026-09-14 07:49:08.110469372 +0000 UTC m=+50.566919358 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/649d1d31-0095-4b78-89e6-cecf97d8d72a-metrics-tls") pod "dns-default-mfdkf" (UID: "649d1d31-0095-4b78-89e6-cecf97d8d72a") : secret "dns-default-metrics-tls" not found
Sep 14 07:49:00.172389 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:00.172350    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-z8xz5_f2032204-d7ba-45f2-81d3-69271762729b/graceful-termination/0.log"
Sep 14 07:49:00.419424 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:00.419337    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-ltmm5" event={"ID":"b73a3fe9-053a-41b9-b85b-93dace0cf9a2","Type":"ContainerStarted","Data":"1e94d6dc165870dddf20e274d304bc6783ec7745e51ef79832d391a43c2d13bd"}
Sep 14 07:49:00.435967 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:00.435899    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-ltmm5" podStartSLOduration=2.129530582 podStartE2EDuration="4.435879537s" totalImagesPullingTime="2.306348955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:00.435651977 +0000 UTC m=+42.892102006" watchObservedRunningTime="2026-09-14 07:49:00.435879537 +0000 UTC m=+42.892329523"
Sep 14 07:49:00.487401 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:00.487361    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:00.588938 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:00.588890    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:00.720884 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:00.720786    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:00.821695 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:00.821660    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:00.922070 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:00.922024    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:01.035242 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:01.035139    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:01.192120 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:01.192075    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:02.388047 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:02.388004    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:02.618790 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:02.618743    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:02.782693 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:02.782639    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:02.830159 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:02.830122    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c2d43af-8e1f-4a7a-a94d-195c15988364-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:49:02.830159 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:02.830164    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2d43af-8e1f-4a7a-a94d-195c15988364-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:49:02.830346 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:49:02.830269    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:49:02.830346 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:49:02.830305    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:02.830405 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:49:02.830349    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c2d43af-8e1f-4a7a-a94d-195c15988364-kube-rbac-proxy-cm podName:6c2d43af-8e1f-4a7a-a94d-195c15988364 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:10.830327804 +0000 UTC m=+53.286777788 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c2d43af-8e1f-4a7a-a94d-195c15988364-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9xx2k" (UID: "6c2d43af-8e1f-4a7a-a94d-195c15988364") : configmap "kube-rbac-proxy" not found
Sep 14 07:49:02.830405 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:49:02.830369    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c2d43af-8e1f-4a7a-a94d-195c15988364-insights-runtime-extractor-tls podName:6c2d43af-8e1f-4a7a-a94d-195c15988364 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:10.830359693 +0000 UTC m=+53.286809667 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c2d43af-8e1f-4a7a-a94d-195c15988364-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9xx2k" (UID: "6c2d43af-8e1f-4a7a-a94d-195c15988364") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:02.979543 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:02.979488    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:03.176693 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:03.176594    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:03.894577 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:03.894545    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:03.994679 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:03.994642    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:04.170740 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:04.170630    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:04.277292 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:04.277256    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:04.387014 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:04.386961    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:04.775877 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:04.775847    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:04.978872 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:04.978836    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:05.064299 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:05.064199    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:05.377304 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:05.377209    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:05.978148 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:05.978097    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:06.938123 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:06.938084    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:07.037297 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:07.037251    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:07.373311 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:07.373192    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:07.660289 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:07.660198    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:08.172437 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:08.172398    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/649d1d31-0095-4b78-89e6-cecf97d8d72a-metrics-tls\") pod \"dns-default-mfdkf\" (UID: \"649d1d31-0095-4b78-89e6-cecf97d8d72a\") " pod="openshift-dns/dns-default-mfdkf"
Sep 14 07:49:08.174765 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:08.174739    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/649d1d31-0095-4b78-89e6-cecf97d8d72a-metrics-tls\") pod \"dns-default-mfdkf\" (UID: \"649d1d31-0095-4b78-89e6-cecf97d8d72a\") " pod="openshift-dns/dns-default-mfdkf"
Sep 14 07:49:08.179566 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:08.179525    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:08.296134 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:08.296097    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mfdkf"
Sep 14 07:49:08.387203 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:08.387163    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:08.579667 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:08.579634    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mfdkf"]
Sep 14 07:49:08.774450 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:08.774406    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:08.978547 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:08.978493    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:09.376653 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:09.376574    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:09.448642 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:09.448607    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mfdkf" event={"ID":"649d1d31-0095-4b78-89e6-cecf97d8d72a","Type":"ContainerStarted","Data":"78da2d47b5e239caf88391397e499d739a5c0dd8756138767a96f87d6b4b314d"}
Sep 14 07:49:09.975603 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:09.975564    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:10.453642 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:10.453606    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mfdkf" event={"ID":"649d1d31-0095-4b78-89e6-cecf97d8d72a","Type":"ContainerStarted","Data":"6c097d1c3712958b7fdd898577cd74d2a9c5cbc105df55f21a8553f0c83f93b5"}
Sep 14 07:49:10.454046 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:10.453665    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mfdkf" event={"ID":"649d1d31-0095-4b78-89e6-cecf97d8d72a","Type":"ContainerStarted","Data":"0b05091289734c186f9350e89ec925e79050f00da07862e3163e326f4e752d3e"}
Sep 14 07:49:10.454046 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:10.453731    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mfdkf"
Sep 14 07:49:10.471987 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:10.471934    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mfdkf" podStartSLOduration=16.868413263 podStartE2EDuration="18.471914947s" totalImagesPullingTime="1.603501684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:10.47141695 +0000 UTC m=+52.927866968" watchObservedRunningTime="2026-09-14 07:49:10.471914947 +0000 UTC m=+52.928364963"
Sep 14 07:49:10.587474 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:10.587382    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:10.773589 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:10.773546    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:10.892093 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:10.892000    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c2d43af-8e1f-4a7a-a94d-195c15988364-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:49:10.892093 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:10.892043    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2d43af-8e1f-4a7a-a94d-195c15988364-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:49:10.892574 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:10.892543    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c2d43af-8e1f-4a7a-a94d-195c15988364-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:49:10.894369 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:10.894351    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2d43af-8e1f-4a7a-a94d-195c15988364-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9xx2k\" (UID: \"6c2d43af-8e1f-4a7a-a94d-195c15988364\") " pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:49:10.943234 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:10.943197    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9xx2k"
Sep 14 07:49:10.979039 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:10.978987    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:11.180572 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:11.180465    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:11.212676 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:11.212642    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9xx2k"]
Sep 14 07:49:11.216341 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:49:11.216303    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c2d43af_8e1f_4a7a_a94d_195c15988364.slice/crio-d0409b257793d60d9d235a529faa4bfbc7dc4d64d32bde07f74516ec75bc488b WatchSource:0}: Error finding container d0409b257793d60d9d235a529faa4bfbc7dc4d64d32bde07f74516ec75bc488b: Status 404 returned error can't find the container with id d0409b257793d60d9d235a529faa4bfbc7dc4d64d32bde07f74516ec75bc488b
Sep 14 07:49:11.458601 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:11.458561    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9xx2k" event={"ID":"6c2d43af-8e1f-4a7a-a94d-195c15988364","Type":"ContainerStarted","Data":"99c6eb0e77ee0b8a55313bd0adbdd3563314f929da3a7e4a51098639173fc617"}
Sep 14 07:49:11.458966 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:11.458631    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9xx2k" event={"ID":"6c2d43af-8e1f-4a7a-a94d-195c15988364","Type":"ContainerStarted","Data":"d0409b257793d60d9d235a529faa4bfbc7dc4d64d32bde07f74516ec75bc488b"}
Sep 14 07:49:12.192548 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:12.192491    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:12.463712 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:12.463680    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9xx2k" event={"ID":"6c2d43af-8e1f-4a7a-a94d-195c15988364","Type":"ContainerStarted","Data":"fc368cd0fe5e5557e17cf6cd48563a52cbb429742da3054099450f754aa6db5d"}
Sep 14 07:49:13.469320 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:13.469286    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9xx2k" event={"ID":"6c2d43af-8e1f-4a7a-a94d-195c15988364","Type":"ContainerStarted","Data":"ec4a90d8462cf6a4c652d41f21e0d54ee823057da6754daee39ea8c7a40ed42d"}
Sep 14 07:49:13.491281 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:13.491231    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9xx2k" podStartSLOduration=16.45023243 podStartE2EDuration="18.491215542s" totalImagesPullingTime="2.040983112s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:13.48990277 +0000 UTC m=+55.946352761" watchObservedRunningTime="2026-09-14 07:49:13.491215542 +0000 UTC m=+55.947665529"
Sep 14 07:49:14.544482 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:14.544419    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:14.646026 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:14.645987    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:14.844222 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:14.844142    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:15.211564 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:15.211526    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:15.612556 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:15.612450    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:15.672151 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:15.672113    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:16.008857 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:16.008826    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:16.063559 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:16.063526    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:16.428467 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:16.428390    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:16.639426 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:16.639395    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:17.060998 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:17.060967    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:17.793142 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:17.793110    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"]
Sep 14 07:49:17.795749 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:17.795732    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"
Sep 14 07:49:17.800191 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:17.800168    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 14 07:49:17.800294 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:17.800233    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 14 07:49:17.800294 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:17.800276    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 14 07:49:17.800403 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:17.800388    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 14 07:49:17.800537 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:17.800523    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 14 07:49:17.802140 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:17.802124    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 14 07:49:17.827880 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:17.827857    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 14 07:49:17.828752 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:17.828734    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"]
Sep 14 07:49:17.928131 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:17.928085    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:17.945086 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:17.945054    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c5752e1b-291d-4402-80ae-f6a8561082d7-ca\") pod \"cluster-proxy-proxy-agent-588c7564db-66jdk\" (UID: \"c5752e1b-291d-4402-80ae-f6a8561082d7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"
Sep 14 07:49:17.945266 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:17.945104    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c5752e1b-291d-4402-80ae-f6a8561082d7-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-588c7564db-66jdk\" (UID: \"c5752e1b-291d-4402-80ae-f6a8561082d7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"
Sep 14 07:49:17.945266 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:17.945133    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c5752e1b-291d-4402-80ae-f6a8561082d7-hub\") pod \"cluster-proxy-proxy-agent-588c7564db-66jdk\" (UID: \"c5752e1b-291d-4402-80ae-f6a8561082d7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"
Sep 14 07:49:17.945266 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:17.945160    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5752e1b-291d-4402-80ae-f6a8561082d7-additional-service-ca\") pod \"cluster-proxy-proxy-agent-588c7564db-66jdk\" (UID: \"c5752e1b-291d-4402-80ae-f6a8561082d7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"
Sep 14 07:49:17.945266 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:17.945184    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzldm\" (UniqueName: \"kubernetes.io/projected/c5752e1b-291d-4402-80ae-f6a8561082d7-kube-api-access-qzldm\") pod \"cluster-proxy-proxy-agent-588c7564db-66jdk\" (UID: \"c5752e1b-291d-4402-80ae-f6a8561082d7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"
Sep 14 07:49:17.945446 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:17.945306    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c5752e1b-291d-4402-80ae-f6a8561082d7-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-588c7564db-66jdk\" (UID: \"c5752e1b-291d-4402-80ae-f6a8561082d7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"
Sep 14 07:49:18.018087 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.018053    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:18.046066 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.045993    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c5752e1b-291d-4402-80ae-f6a8561082d7-hub\") pod \"cluster-proxy-proxy-agent-588c7564db-66jdk\" (UID: \"c5752e1b-291d-4402-80ae-f6a8561082d7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"
Sep 14 07:49:18.046066 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.046026    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5752e1b-291d-4402-80ae-f6a8561082d7-additional-service-ca\") pod \"cluster-proxy-proxy-agent-588c7564db-66jdk\" (UID: \"c5752e1b-291d-4402-80ae-f6a8561082d7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"
Sep 14 07:49:18.046066 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.046047    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzldm\" (UniqueName: \"kubernetes.io/projected/c5752e1b-291d-4402-80ae-f6a8561082d7-kube-api-access-qzldm\") pod \"cluster-proxy-proxy-agent-588c7564db-66jdk\" (UID: \"c5752e1b-291d-4402-80ae-f6a8561082d7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"
Sep 14 07:49:18.046297 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.046125    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c5752e1b-291d-4402-80ae-f6a8561082d7-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-588c7564db-66jdk\" (UID: \"c5752e1b-291d-4402-80ae-f6a8561082d7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"
Sep 14 07:49:18.046297 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.046174    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c5752e1b-291d-4402-80ae-f6a8561082d7-ca\") pod \"cluster-proxy-proxy-agent-588c7564db-66jdk\" (UID: \"c5752e1b-291d-4402-80ae-f6a8561082d7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"
Sep 14 07:49:18.046297 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.046202    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c5752e1b-291d-4402-80ae-f6a8561082d7-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-588c7564db-66jdk\" (UID: \"c5752e1b-291d-4402-80ae-f6a8561082d7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"
Sep 14 07:49:18.048239 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.048209    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 14 07:49:18.049009 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.048992    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 14 07:49:18.049531 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.049518    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 14 07:49:18.049695 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.049676    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 14 07:49:18.049774 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.049726    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 14 07:49:18.057626 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.057602    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5752e1b-291d-4402-80ae-f6a8561082d7-additional-service-ca\") pod \"cluster-proxy-proxy-agent-588c7564db-66jdk\" (UID: \"c5752e1b-291d-4402-80ae-f6a8561082d7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"
Sep 14 07:49:18.057773 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.057751    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 14 07:49:18.059445 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.059420    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c5752e1b-291d-4402-80ae-f6a8561082d7-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-588c7564db-66jdk\" (UID: \"c5752e1b-291d-4402-80ae-f6a8561082d7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"
Sep 14 07:49:18.059628 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.059610    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c5752e1b-291d-4402-80ae-f6a8561082d7-hub\") pod \"cluster-proxy-proxy-agent-588c7564db-66jdk\" (UID: \"c5752e1b-291d-4402-80ae-f6a8561082d7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"
Sep 14 07:49:18.059685 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.059637    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c5752e1b-291d-4402-80ae-f6a8561082d7-ca\") pod \"cluster-proxy-proxy-agent-588c7564db-66jdk\" (UID: \"c5752e1b-291d-4402-80ae-f6a8561082d7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"
Sep 14 07:49:18.059805 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.059788    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c5752e1b-291d-4402-80ae-f6a8561082d7-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-588c7564db-66jdk\" (UID: \"c5752e1b-291d-4402-80ae-f6a8561082d7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"
Sep 14 07:49:18.067313 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.067291    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzldm\" (UniqueName: \"kubernetes.io/projected/c5752e1b-291d-4402-80ae-f6a8561082d7-kube-api-access-qzldm\") pod \"cluster-proxy-proxy-agent-588c7564db-66jdk\" (UID: \"c5752e1b-291d-4402-80ae-f6a8561082d7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"
Sep 14 07:49:18.110436 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.110373    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:18.120557 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.120534    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 14 07:49:18.128026 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.128005    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"
Sep 14 07:49:18.328751 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.328654    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:18.382102 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.382078    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fw2m9"
Sep 14 07:49:18.400490 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.400457    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk"]
Sep 14 07:49:18.403842 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:49:18.403820    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5752e1b_291d_4402_80ae_f6a8561082d7.slice/crio-b4bd587e780b56917f22e30e2bb01e8e3bf2e2d9fe8b1af4d2104ea5c61a15e0 WatchSource:0}: Error finding container b4bd587e780b56917f22e30e2bb01e8e3bf2e2d9fe8b1af4d2104ea5c61a15e0: Status 404 returned error can't find the container with id b4bd587e780b56917f22e30e2bb01e8e3bf2e2d9fe8b1af4d2104ea5c61a15e0
Sep 14 07:49:18.487292 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.487259    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk" event={"ID":"c5752e1b-291d-4402-80ae-f6a8561082d7","Type":"ContainerStarted","Data":"b4bd587e780b56917f22e30e2bb01e8e3bf2e2d9fe8b1af4d2104ea5c61a15e0"}
Sep 14 07:49:18.701647 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.701611    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:18.782913 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:18.782878    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:19.084918 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:19.084817    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:19.148724 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:19.148692    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:20.262929 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:20.262823    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:20.461734 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:20.461242    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mfdkf"
Sep 14 07:49:20.468416 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:20.468375    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:20.569864 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:20.569778    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:22.503874 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:22.503840    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk" event={"ID":"c5752e1b-291d-4402-80ae-f6a8561082d7","Type":"ContainerStarted","Data":"37bc033ca50c3004bcb28d2f858bddeca01f0702cda6f52c5beb957ec12b208d"}
Sep 14 07:49:22.504249 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:22.503884    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk" event={"ID":"c5752e1b-291d-4402-80ae-f6a8561082d7","Type":"ContainerStarted","Data":"873be25cd0d545ee1cb8095044e86a70f05f0fdc9110700ec6cfd572bf921d73"}
Sep 14 07:49:22.504249 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:22.503897    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk" event={"ID":"c5752e1b-291d-4402-80ae-f6a8561082d7","Type":"ContainerStarted","Data":"e6c93514e6d9f9a2b7116e2fa7691ed36b998935405dd93919850a145354b195"}
Sep 14 07:49:22.523878 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:22.523821    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-588c7564db-66jdk" podStartSLOduration=2.095520567 podStartE2EDuration="5.523805825s" totalImagesPullingTime="3.428285258s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:22.523412977 +0000 UTC m=+64.979862969" watchObservedRunningTime="2026-09-14 07:49:22.523805825 +0000 UTC m=+64.980255819"
Sep 14 07:49:23.691332 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:23.691293    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs\") pod \"network-metrics-daemon-4mfj7\" (UID: \"fd5d1c55-8829-4084-9b68-36cac615651b\") " pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:49:23.691332 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:23.691343    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret\") pod \"global-pull-secret-syncer-xbb69\" (UID: \"4b4a6526-1631-4af9-9a20-cf80f53d3eac\") " pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:49:23.693707 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:23.693683    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 07:49:23.693707 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:23.693694    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 07:49:23.704347 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:23.704322    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fd5d1c55-8829-4084-9b68-36cac615651b-metrics-certs\") pod \"network-metrics-daemon-4mfj7\" (UID: \"fd5d1c55-8829-4084-9b68-36cac615651b\") " pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:49:23.704452 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:23.704430    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b4a6526-1631-4af9-9a20-cf80f53d3eac-original-pull-secret\") pod \"global-pull-secret-syncer-xbb69\" (UID: \"4b4a6526-1631-4af9-9a20-cf80f53d3eac\") " pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:49:23.821135 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:23.821101    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bd4sd\""
Sep 14 07:49:23.829909 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:23.829883    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4mfj7"
Sep 14 07:49:23.831726 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:23.831705    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xbb69"
Sep 14 07:49:23.892464 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:23.892433    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77fds\" (UniqueName: \"kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds\") pod \"network-check-target-v7dql\" (UID: \"f3e50313-f82c-47b4-a66b-0bd7f9e227f4\") " pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:49:23.895330 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:23.895302    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 07:49:23.905853 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:23.905833    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 07:49:23.916184 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:23.916165    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77fds\" (UniqueName: \"kubernetes.io/projected/f3e50313-f82c-47b4-a66b-0bd7f9e227f4-kube-api-access-77fds\") pod \"network-check-target-v7dql\" (UID: \"f3e50313-f82c-47b4-a66b-0bd7f9e227f4\") " pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:49:24.110934 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:49:24.110890    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd5d1c55_8829_4084_9b68_36cac615651b.slice/crio-761bc74f08d4bc1a908846a97b88c3f45aa964a40ac3dc477b3854d90736be81 WatchSource:0}: Error finding container 761bc74f08d4bc1a908846a97b88c3f45aa964a40ac3dc477b3854d90736be81: Status 404 returned error can't find the container with id 761bc74f08d4bc1a908846a97b88c3f45aa964a40ac3dc477b3854d90736be81
Sep 14 07:49:24.111463 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:24.111443    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4mfj7"]
Sep 14 07:49:24.111566 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:24.111494    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xbb69"]
Sep 14 07:49:24.113689 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:49:24.113668    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4a6526_1631_4af9_9a20_cf80f53d3eac.slice/crio-73d8e26cabfea65e32d726e965a01e9b8387e646a8e4e6b967057a4007ba5ab9 WatchSource:0}: Error finding container 73d8e26cabfea65e32d726e965a01e9b8387e646a8e4e6b967057a4007ba5ab9: Status 404 returned error can't find the container with id 73d8e26cabfea65e32d726e965a01e9b8387e646a8e4e6b967057a4007ba5ab9
Sep 14 07:49:24.128435 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:24.128411    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qxrn8\""
Sep 14 07:49:24.136868 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:24.136847    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:49:24.404273 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:24.404193    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-v7dql"]
Sep 14 07:49:24.408654 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:49:24.408627    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3e50313_f82c_47b4_a66b_0bd7f9e227f4.slice/crio-7538de0d4b203461e59eb59a0f9dc57d7b2405c2f70e4b6d34131dbfa302e1c4 WatchSource:0}: Error finding container 7538de0d4b203461e59eb59a0f9dc57d7b2405c2f70e4b6d34131dbfa302e1c4: Status 404 returned error can't find the container with id 7538de0d4b203461e59eb59a0f9dc57d7b2405c2f70e4b6d34131dbfa302e1c4
Sep 14 07:49:24.512030 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:24.511995    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-v7dql" event={"ID":"f3e50313-f82c-47b4-a66b-0bd7f9e227f4","Type":"ContainerStarted","Data":"7538de0d4b203461e59eb59a0f9dc57d7b2405c2f70e4b6d34131dbfa302e1c4"}
Sep 14 07:49:24.513116 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:24.513087    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xbb69" event={"ID":"4b4a6526-1631-4af9-9a20-cf80f53d3eac","Type":"ContainerStarted","Data":"73d8e26cabfea65e32d726e965a01e9b8387e646a8e4e6b967057a4007ba5ab9"}
Sep 14 07:49:24.514189 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:24.514169    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4mfj7" event={"ID":"fd5d1c55-8829-4084-9b68-36cac615651b","Type":"ContainerStarted","Data":"761bc74f08d4bc1a908846a97b88c3f45aa964a40ac3dc477b3854d90736be81"}
Sep 14 07:49:24.854373 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:24.854327    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:24.954952 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:24.954914    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:25.039884 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:25.039785    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:25.452033 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:25.451995    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:25.525291 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:25.525254    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4mfj7" event={"ID":"fd5d1c55-8829-4084-9b68-36cac615651b","Type":"ContainerStarted","Data":"e75a46e9af5d8d7862a4033c651dd7851b2e16e86ee4bbb2249cd62be2d14412"}
Sep 14 07:49:26.052612 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:26.052572    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:26.106393 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:26.106354    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:26.531550 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:26.531512    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4mfj7" event={"ID":"fd5d1c55-8829-4084-9b68-36cac615651b","Type":"ContainerStarted","Data":"518d6b8521ff2a61ccc9bf9c51d7b6a525d7a1be4c73498b72028102d5fc126e"}
Sep 14 07:49:26.551174 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:26.551101    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4mfj7" podStartSLOduration=66.283456059 podStartE2EDuration="1m7.551078404s" totalImagesPullingTime="1.267622345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:26.550147331 +0000 UTC m=+69.006597327" watchObservedRunningTime="2026-09-14 07:49:26.551078404 +0000 UTC m=+69.007528398"
Sep 14 07:49:26.646631 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:26.646560    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:26.706569 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:26.706533    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:28.470286 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:28.470248    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:28.523827 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:28.523769    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:29.546539 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:29.546465    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-v7dql" event={"ID":"f3e50313-f82c-47b4-a66b-0bd7f9e227f4","Type":"ContainerStarted","Data":"b25ebdb2a83b52bf924b96023fa6db3505cbb4daaa072a54a853875664a690f4"}
Sep 14 07:49:29.547043 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:29.546710    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:49:29.548089 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:29.548061    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xbb69" event={"ID":"4b4a6526-1631-4af9-9a20-cf80f53d3eac","Type":"ContainerStarted","Data":"63786c1db46201f96f7db99c2d52a4a0f2968703ef7543e6e7117aac9a0fad8f"}
Sep 14 07:49:29.576426 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:29.576368    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xbb69" podStartSLOduration=65.320501292 podStartE2EDuration="1m10.576347629s" totalImagesPullingTime="5.255846337s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:29.575756497 +0000 UTC m=+72.032206491" watchObservedRunningTime="2026-09-14 07:49:29.576347629 +0000 UTC m=+72.032797623"
Sep 14 07:49:29.576626 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:29.576568    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-v7dql" podStartSLOduration=65.613872422 podStartE2EDuration="1m10.576557931s" totalImagesPullingTime="4.962685509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:29.561634403 +0000 UTC m=+72.018084399" watchObservedRunningTime="2026-09-14 07:49:29.576557931 +0000 UTC m=+72.033007924"
Sep 14 07:49:30.058699 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:30.058658    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:30.134312 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:30.134282    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:30.247481 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:30.247443    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:31.477321 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:31.477288    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:31.553399 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:31.553363    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:32.854962 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:32.854925    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:32.955325 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:32.955290    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:35.660288 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:35.660249    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:35.763216 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:35.763155    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:36.346095 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:36.346056    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:36.445458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:36.445404    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:38.495768 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:38.495722    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:38.581029 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:38.580996    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:40.461621 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:40.461474    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:40.561911 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:40.561876    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:40.662722 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:40.662681    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:41.517594 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:41.517557    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:41.616209 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:41.616173    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:41.719186 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:41.719140    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:41.816621 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:41.816490    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:44.215125 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.215086    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:44.418129 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.418091    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:44.470312 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.470227    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9sqqx"]
Sep 14 07:49:44.476684 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.476658    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.479006 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.478787    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 14 07:49:44.479006 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.478841    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 14 07:49:44.479006 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.478940    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 14 07:49:44.479603 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.479309    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 14 07:49:44.479603 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.479321    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 14 07:49:44.479603 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.479329    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 14 07:49:44.479603 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.479361    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6jx2x\""
Sep 14 07:49:44.506622 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.506584    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:44.540800 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.540761    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/92a3911a-36bc-4793-bfad-d598640f14ef-node-exporter-accelerators-collector-config\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.541010 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.540818    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/92a3911a-36bc-4793-bfad-d598640f14ef-node-exporter-textfile\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.541010 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.540883    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2qsb\" (UniqueName: \"kubernetes.io/projected/92a3911a-36bc-4793-bfad-d598640f14ef-kube-api-access-g2qsb\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.541010 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.540913    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/92a3911a-36bc-4793-bfad-d598640f14ef-node-exporter-wtmp\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.541010 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.540958    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92a3911a-36bc-4793-bfad-d598640f14ef-sys\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.541010 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.540983    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92a3911a-36bc-4793-bfad-d598640f14ef-metrics-client-ca\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.541265 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.541024    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/92a3911a-36bc-4793-bfad-d598640f14ef-root\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.541265 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.541091    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/92a3911a-36bc-4793-bfad-d598640f14ef-node-exporter-tls\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.541265 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.541134    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/92a3911a-36bc-4793-bfad-d598640f14ef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.572231 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.572197    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:44.642523 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.642465    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92a3911a-36bc-4793-bfad-d598640f14ef-sys\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.642718 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.642534    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92a3911a-36bc-4793-bfad-d598640f14ef-metrics-client-ca\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.642718 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.642568    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/92a3911a-36bc-4793-bfad-d598640f14ef-root\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.642718 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.642592    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/92a3911a-36bc-4793-bfad-d598640f14ef-node-exporter-tls\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.642718 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.642606    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/92a3911a-36bc-4793-bfad-d598640f14ef-sys\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.642718 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.642617    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/92a3911a-36bc-4793-bfad-d598640f14ef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.642718 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.642692    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/92a3911a-36bc-4793-bfad-d598640f14ef-root\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.642718 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.642706    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/92a3911a-36bc-4793-bfad-d598640f14ef-node-exporter-accelerators-collector-config\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.643116 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:49:44.642728    2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 14 07:49:44.643116 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.642749    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/92a3911a-36bc-4793-bfad-d598640f14ef-node-exporter-textfile\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.643116 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:49:44.642800    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92a3911a-36bc-4793-bfad-d598640f14ef-node-exporter-tls podName:92a3911a-36bc-4793-bfad-d598640f14ef nodeName:}" failed. No retries permitted until 2026-09-14 07:49:45.142780132 +0000 UTC m=+87.599230109 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/92a3911a-36bc-4793-bfad-d598640f14ef-node-exporter-tls") pod "node-exporter-9sqqx" (UID: "92a3911a-36bc-4793-bfad-d598640f14ef") : secret "node-exporter-tls" not found
Sep 14 07:49:44.643116 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.642834    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2qsb\" (UniqueName: \"kubernetes.io/projected/92a3911a-36bc-4793-bfad-d598640f14ef-kube-api-access-g2qsb\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.643116 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.642868    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/92a3911a-36bc-4793-bfad-d598640f14ef-node-exporter-wtmp\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.643116 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.643011    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/92a3911a-36bc-4793-bfad-d598640f14ef-node-exporter-wtmp\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.643416 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.643123    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/92a3911a-36bc-4793-bfad-d598640f14ef-node-exporter-textfile\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.643600 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.643579    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/92a3911a-36bc-4793-bfad-d598640f14ef-node-exporter-accelerators-collector-config\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.643993 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.643963    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/92a3911a-36bc-4793-bfad-d598640f14ef-metrics-client-ca\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.645716 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.645692    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/92a3911a-36bc-4793-bfad-d598640f14ef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.651317 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.651275    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2qsb\" (UniqueName: \"kubernetes.io/projected/92a3911a-36bc-4793-bfad-d598640f14ef-kube-api-access-g2qsb\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:44.672734 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.672694    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:44.773777 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.773689    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:44.885175 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.885137    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:44.986138 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:44.986098    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:45.085933 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.085830    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:45.148041 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.148006    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/92a3911a-36bc-4793-bfad-d598640f14ef-node-exporter-tls\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:45.150322 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.150287    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/92a3911a-36bc-4793-bfad-d598640f14ef-node-exporter-tls\") pod \"node-exporter-9sqqx\" (UID: \"92a3911a-36bc-4793-bfad-d598640f14ef\") " pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:45.188563 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.188524    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:45.289056 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.289021    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:45.388568 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.388470    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9sqqx"
Sep 14 07:49:45.400215 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:49:45.400180    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92a3911a_36bc_4793_bfad_d598640f14ef.slice/crio-62f64bdb46c28af88d579b754bf8e38de9479cfc091c88a2cd1952bbc793bac5 WatchSource:0}: Error finding container 62f64bdb46c28af88d579b754bf8e38de9479cfc091c88a2cd1952bbc793bac5: Status 404 returned error can't find the container with id 62f64bdb46c28af88d579b754bf8e38de9479cfc091c88a2cd1952bbc793bac5
Sep 14 07:49:45.439254 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.439214    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:45.538127 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.538094    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:45.539609 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.539580    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 14 07:49:45.543203 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.543186    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.545372 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.545352    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 14 07:49:45.545517 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.545473    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 14 07:49:45.545517 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.545485    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 14 07:49:45.545626 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.545595    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 14 07:49:45.545863 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.545844    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 14 07:49:45.545948 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.545883    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 14 07:49:45.545948 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.545931    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-59tdq\""
Sep 14 07:49:45.546040 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.545970    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 14 07:49:45.547753 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.547732    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 14 07:49:45.547896 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.547877    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 14 07:49:45.562482 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.562455    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 14 07:49:45.599970 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.599935    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9sqqx" event={"ID":"92a3911a-36bc-4793-bfad-d598640f14ef","Type":"ContainerStarted","Data":"62f64bdb46c28af88d579b754bf8e38de9479cfc091c88a2cd1952bbc793bac5"}
Sep 14 07:49:45.651525 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.651422    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-config-volume\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.651525 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.651466    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.651525 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.651520    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c023ea1-de79-46d0-963e-583ef335e625-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.651755 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.651560    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.651755 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.651586    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c023ea1-de79-46d0-963e-583ef335e625-config-out\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.651755 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.651611    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.651755 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.651638    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb6xf\" (UniqueName: \"kubernetes.io/projected/5c023ea1-de79-46d0-963e-583ef335e625-kube-api-access-cb6xf\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.651755 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.651725    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.651914 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.651761    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c023ea1-de79-46d0-963e-583ef335e625-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.651914 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.651783    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.651914 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.651811    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5c023ea1-de79-46d0-963e-583ef335e625-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.651914 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.651829    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c023ea1-de79-46d0-963e-583ef335e625-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.651914 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.651853    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-web-config\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.752586 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.752547    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-config-volume\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.752586 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.752590    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.752823 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.752615    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c023ea1-de79-46d0-963e-583ef335e625-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.752823 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.752640    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.752823 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.752665    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c023ea1-de79-46d0-963e-583ef335e625-config-out\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.752823 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.752686    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.752823 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.752765    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb6xf\" (UniqueName: \"kubernetes.io/projected/5c023ea1-de79-46d0-963e-583ef335e625-kube-api-access-cb6xf\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.753080 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.752825    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.753080 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.752863    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c023ea1-de79-46d0-963e-583ef335e625-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.753080 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.752887    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.754023 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.753544    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5c023ea1-de79-46d0-963e-583ef335e625-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.754023 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.753601    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c023ea1-de79-46d0-963e-583ef335e625-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.754023 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.753652    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-web-config\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.754832 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.754712    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c023ea1-de79-46d0-963e-583ef335e625-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.755091 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.755035    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5c023ea1-de79-46d0-963e-583ef335e625-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.755607 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.755586    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c023ea1-de79-46d0-963e-583ef335e625-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.755900 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.755873    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.756071 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.756050    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.756198 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.756128    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c023ea1-de79-46d0-963e-583ef335e625-config-out\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.756607 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.756582    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.756702 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.756621    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-config-volume\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.757254 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.757213    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.757344 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.757314    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.757486 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.757471    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c023ea1-de79-46d0-963e-583ef335e625-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.757623 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.757608    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-web-config\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.758214 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.758191    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:45.766295 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.766275    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb6xf\" (UniqueName: \"kubernetes.io/projected/5c023ea1-de79-46d0-963e-583ef335e625-kube-api-access-cb6xf\") pod \"alertmanager-main-0\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:45.852602 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:45.852549    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:49:46.140001 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:46.139968    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 14 07:49:46.143591 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:49:46.143560    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c023ea1_de79_46d0_963e_583ef335e625.slice/crio-e55d790504d85aa5f04750c9ef7510792700877d7369b0f57a2b295663c730a1 WatchSource:0}: Error finding container e55d790504d85aa5f04750c9ef7510792700877d7369b0f57a2b295663c730a1: Status 404 returned error can't find the container with id e55d790504d85aa5f04750c9ef7510792700877d7369b0f57a2b295663c730a1
Sep 14 07:49:46.604815 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:46.604781    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c023ea1-de79-46d0-963e-583ef335e625","Type":"ContainerStarted","Data":"e55d790504d85aa5f04750c9ef7510792700877d7369b0f57a2b295663c730a1"}
Sep 14 07:49:46.606433 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:46.606409    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9sqqx" event={"ID":"92a3911a-36bc-4793-bfad-d598640f14ef","Type":"ContainerStarted","Data":"a49282ad5230571bbe6e1c49ec5b7140d66290293faf95d4ef07b00b78c16e2d"}
Sep 14 07:49:47.523071 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.522980    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:47.558033 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.557996    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"]
Sep 14 07:49:47.563200 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.563163    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.566573 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.566519    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 14 07:49:47.566573 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.566545    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 14 07:49:47.566840 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.566826    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 14 07:49:47.566920 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.566825    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2nln1riv9nskh\""
Sep 14 07:49:47.568317 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.568296    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-tpbnf\""
Sep 14 07:49:47.568423 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.568301    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 14 07:49:47.568484 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.568433    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 14 07:49:47.572448 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.572426    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"]
Sep 14 07:49:47.591523 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.591476    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:47.611880 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.611852    2562 generic.go:356] "Generic (PLEG): container finished" podID="5c023ea1-de79-46d0-963e-583ef335e625" containerID="96550f091cb8f8a634783ff540bab4443b00e5f3d46a3af9f634beba6cd4a34c" exitCode=0
Sep 14 07:49:47.612287 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.611933    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c023ea1-de79-46d0-963e-583ef335e625","Type":"ContainerDied","Data":"96550f091cb8f8a634783ff540bab4443b00e5f3d46a3af9f634beba6cd4a34c"}
Sep 14 07:49:47.613808 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.613744    2562 generic.go:356] "Generic (PLEG): container finished" podID="92a3911a-36bc-4793-bfad-d598640f14ef" containerID="a49282ad5230571bbe6e1c49ec5b7140d66290293faf95d4ef07b00b78c16e2d" exitCode=0
Sep 14 07:49:47.613808 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.613802    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9sqqx" event={"ID":"92a3911a-36bc-4793-bfad-d598640f14ef","Type":"ContainerDied","Data":"a49282ad5230571bbe6e1c49ec5b7140d66290293faf95d4ef07b00b78c16e2d"}
Sep 14 07:49:47.669984 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.669950    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/173d0d44-527b-47d0-99e2-3ffb1058ba31-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.670230 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.670202    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/173d0d44-527b-47d0-99e2-3ffb1058ba31-metrics-client-ca\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.670352 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.670333    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/173d0d44-527b-47d0-99e2-3ffb1058ba31-secret-grpc-tls\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.670417 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.670362    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/173d0d44-527b-47d0-99e2-3ffb1058ba31-secret-thanos-querier-tls\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.670417 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.670410    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p76p4\" (UniqueName: \"kubernetes.io/projected/173d0d44-527b-47d0-99e2-3ffb1058ba31-kube-api-access-p76p4\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.670546 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.670484    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/173d0d44-527b-47d0-99e2-3ffb1058ba31-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.670648 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.670621    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/173d0d44-527b-47d0-99e2-3ffb1058ba31-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.670704 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.670671    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/173d0d44-527b-47d0-99e2-3ffb1058ba31-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.676009 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.675981    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:47.771464 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.771424    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/173d0d44-527b-47d0-99e2-3ffb1058ba31-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.771635 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.771468    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/173d0d44-527b-47d0-99e2-3ffb1058ba31-metrics-client-ca\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.771635 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.771512    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/173d0d44-527b-47d0-99e2-3ffb1058ba31-secret-grpc-tls\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.771742 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.771649    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/173d0d44-527b-47d0-99e2-3ffb1058ba31-secret-thanos-querier-tls\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.771742 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.771706    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p76p4\" (UniqueName: \"kubernetes.io/projected/173d0d44-527b-47d0-99e2-3ffb1058ba31-kube-api-access-p76p4\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.771847 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.771756    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/173d0d44-527b-47d0-99e2-3ffb1058ba31-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.771847 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.771796    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/173d0d44-527b-47d0-99e2-3ffb1058ba31-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.771847 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.771825    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/173d0d44-527b-47d0-99e2-3ffb1058ba31-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.772602 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.772545    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/173d0d44-527b-47d0-99e2-3ffb1058ba31-metrics-client-ca\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.774695 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.774632    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/173d0d44-527b-47d0-99e2-3ffb1058ba31-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.774811 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.774701    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/173d0d44-527b-47d0-99e2-3ffb1058ba31-secret-thanos-querier-tls\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.774811 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.774751    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/173d0d44-527b-47d0-99e2-3ffb1058ba31-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.774961 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.774912    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/173d0d44-527b-47d0-99e2-3ffb1058ba31-secret-grpc-tls\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.775556 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.775524    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/173d0d44-527b-47d0-99e2-3ffb1058ba31-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.775643 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.775590    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/173d0d44-527b-47d0-99e2-3ffb1058ba31-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.780829 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.780797    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p76p4\" (UniqueName: \"kubernetes.io/projected/173d0d44-527b-47d0-99e2-3ffb1058ba31-kube-api-access-p76p4\") pod \"thanos-querier-5c7dc76f84-gdmhf\" (UID: \"173d0d44-527b-47d0-99e2-3ffb1058ba31\") " pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.874292 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.874249    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:47.994256 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:47.994212    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:48.048619 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:48.048527    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:48.159220 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:48.159039    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"]
Sep 14 07:49:48.163434 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:49:48.163402    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod173d0d44_527b_47d0_99e2_3ffb1058ba31.slice/crio-c114320dfaffe7ab0130d433aa976c8fb1f5e1a5646989b14fd9c306ba0e6697 WatchSource:0}: Error finding container c114320dfaffe7ab0130d433aa976c8fb1f5e1a5646989b14fd9c306ba0e6697: Status 404 returned error can't find the container with id c114320dfaffe7ab0130d433aa976c8fb1f5e1a5646989b14fd9c306ba0e6697
Sep 14 07:49:48.579748 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:48.579710    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:48.621061 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:48.621023    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9sqqx" event={"ID":"92a3911a-36bc-4793-bfad-d598640f14ef","Type":"ContainerStarted","Data":"8b0336a79f205de51c6a7ca55e4384dcbe000e51536817624df987d967ce0361"}
Sep 14 07:49:48.621461 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:48.621092    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9sqqx" event={"ID":"92a3911a-36bc-4793-bfad-d598640f14ef","Type":"ContainerStarted","Data":"08c5b8c415303badddf4b31f280c4e16e8d4162453953dbd4fefb904ac33e762"}
Sep 14 07:49:48.622389 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:48.622365    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf" event={"ID":"173d0d44-527b-47d0-99e2-3ffb1058ba31","Type":"ContainerStarted","Data":"c114320dfaffe7ab0130d433aa976c8fb1f5e1a5646989b14fd9c306ba0e6697"}
Sep 14 07:49:48.641199 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:48.641142    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9sqqx" podStartSLOduration=3.459202611 podStartE2EDuration="4.641120346s" totalImagesPullingTime="1.118939573s" totalInitContainerRuntime="62.978162ms" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:48.6400219 +0000 UTC m=+91.096471894" watchObservedRunningTime="2026-09-14 07:49:48.641120346 +0000 UTC m=+91.097570341"
Sep 14 07:49:48.645107 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:48.645072    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:48.702394 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:48.702342    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:48.786454 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:48.786418    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:49.282467 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:49.282358    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:49.362305 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:49.362273    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:49.481135 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:49.481084    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:49.629739 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:49.629700    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c023ea1-de79-46d0-963e-583ef335e625","Type":"ContainerStarted","Data":"342c00dbb22ddffd7c5c13b9fc21d11857d9e2c8436611c7a7a595026a259b8b"}
Sep 14 07:49:49.630163 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:49.629785    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c023ea1-de79-46d0-963e-583ef335e625","Type":"ContainerStarted","Data":"c5fe061eb85cd309b7d54af7ce14c1512d7987939afc3000ee3fbbb3729db9cb"}
Sep 14 07:49:49.630163 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:49.629827    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c023ea1-de79-46d0-963e-583ef335e625","Type":"ContainerStarted","Data":"5346e64ef93847abe21314a25960c99dbed97761eef03d1f0b8e8da72cf020fe"}
Sep 14 07:49:49.630163 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:49.629868    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c023ea1-de79-46d0-963e-583ef335e625","Type":"ContainerStarted","Data":"a0978b2f7e9812c44da51c5f7d91354a6728b45c448654ea6c16d9c036b8c6a7"}
Sep 14 07:49:49.960024 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:49.959967    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:50.056419 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.056388    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:50.638384 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.638303    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c023ea1-de79-46d0-963e-583ef335e625","Type":"ContainerStarted","Data":"b7754e19912481bb5a1dab30cd3bfcb1ede8fe257a8b516896dd0dcd54c9febf"}
Sep 14 07:49:50.641002 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.640972    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf" event={"ID":"173d0d44-527b-47d0-99e2-3ffb1058ba31","Type":"ContainerStarted","Data":"8c72e9d0841945edeb7364b60da040d94b0f9fbf4a585d1d5fe57b4c32801ae2"}
Sep 14 07:49:50.641138 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.641049    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf" event={"ID":"173d0d44-527b-47d0-99e2-3ffb1058ba31","Type":"ContainerStarted","Data":"a35a0be2c488a536b2f3aa1d32206852b0a5e90d8736fd51cbf7cb37e3f56e57"}
Sep 14 07:49:50.641138 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.641092    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf" event={"ID":"173d0d44-527b-47d0-99e2-3ffb1058ba31","Type":"ContainerStarted","Data":"171545d57c68bc737c30ed501c73075520f0776538838dde1550dcc324a2343a"}
Sep 14 07:49:50.830568 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.830531    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:50.848516 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.848460    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 07:49:50.852613 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.852583    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.854921 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.854888    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 14 07:49:50.855037 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.854906    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 14 07:49:50.855037 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.854946    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1ns2lg6g237s5\""
Sep 14 07:49:50.855534 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.855514    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 14 07:49:50.856258 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.856234    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 14 07:49:50.856377 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.856278    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 14 07:49:50.856608 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.856587    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 14 07:49:50.856911 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.856888    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 14 07:49:50.857020 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.856934    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 14 07:49:50.857020 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.856987    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5vtms\""
Sep 14 07:49:50.857122 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.857023    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 14 07:49:50.857358 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.857280    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 14 07:49:50.857358 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.857329    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 14 07:49:50.857575 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.857467    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 14 07:49:50.859835 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.859809    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 14 07:49:50.861998 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.861418    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 14 07:49:50.864983 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.864881    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 14 07:49:50.872994 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.872956    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 07:49:50.897892 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.897818    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-web-config\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.897892 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.897879    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5324fdbc-a7c2-484c-b0a2-641594258899-config-out\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.898071 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.897899    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.898071 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.897916    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.898071 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.897950    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5324fdbc-a7c2-484c-b0a2-641594258899-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.898071 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.897965    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.898071 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.897981    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.898071 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.897998    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.898071 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.898023    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.898414 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.898295    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-config\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.898414 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.898338    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.898414 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.898365    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.898414 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.898397    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.898633 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.898431    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znwtx\" (UniqueName: \"kubernetes.io/projected/5324fdbc-a7c2-484c-b0a2-641594258899-kube-api-access-znwtx\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.898633 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.898476    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.898633 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.898524    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.898633 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.898555    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.898633 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.898585    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.898633 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.898619    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.898861 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.898662    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.999480 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.999458    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-web-config\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.999593 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.999493    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5324fdbc-a7c2-484c-b0a2-641594258899-config-out\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.999593 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.999529    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.999593 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.999560    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.999758 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.999599    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5324fdbc-a7c2-484c-b0a2-641594258899-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.999758 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.999636    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.999758 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.999664    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.999758 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.999695    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.999758 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.999729    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.999996 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.999769    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-config\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.999996 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.999793    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.999996 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.999821    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.999996 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.999853    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.999996 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.999888    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znwtx\" (UniqueName: \"kubernetes.io/projected/5324fdbc-a7c2-484c-b0a2-641594258899-kube-api-access-znwtx\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.999996 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.999937    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:50.999996 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.999967    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.000357 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:50.999999    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.000357 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.000029    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.000357 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.000061    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.000357 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.000098    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.000581 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.000412    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.001716 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.001027    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.002261 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.002085    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.002371 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.002313    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.003277 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.003253    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.003694 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.003666    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.004264 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.004234    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.004625 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.004600    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.005494 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.005450    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.006392 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.006371    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-config\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.006883 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.006856    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-web-config\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.007627 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.007602    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.008041 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.008016    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5324fdbc-a7c2-484c-b0a2-641594258899-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.008221 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.008194    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.008356 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.008198    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.008434 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.008410    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5324fdbc-a7c2-484c-b0a2-641594258899-config-out\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.008595 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.008574    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.008788 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.008760    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.009647 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.009567    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.016086 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.016062    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znwtx\" (UniqueName: \"kubernetes.io/projected/5324fdbc-a7c2-484c-b0a2-641594258899-kube-api-access-znwtx\") pod \"prometheus-k8s-0\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.056693 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.056662    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:51.165650 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.165563    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:51.451900 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.451735    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 07:49:51.454272 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:49:51.454245    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5324fdbc_a7c2_484c_b0a2_641594258899.slice/crio-1cb30023ddd21668fd0c7d11630a66458d2d57d4d1ba213598b28410c6e5ca13 WatchSource:0}: Error finding container 1cb30023ddd21668fd0c7d11630a66458d2d57d4d1ba213598b28410c6e5ca13: Status 404 returned error can't find the container with id 1cb30023ddd21668fd0c7d11630a66458d2d57d4d1ba213598b28410c6e5ca13
Sep 14 07:49:51.648606 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.648570    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c023ea1-de79-46d0-963e-583ef335e625","Type":"ContainerStarted","Data":"8ee39965614a4883e85b2c5b45c3620a804213b89418c3ed8c29d9cd9ee97319"}
Sep 14 07:49:51.651795 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.651766    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf" event={"ID":"173d0d44-527b-47d0-99e2-3ffb1058ba31","Type":"ContainerStarted","Data":"4b441ca3a37ab0c3f22cff6f9bc1bed169998fb4142599604fbdaf7261c98b30"}
Sep 14 07:49:51.652050 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.651828    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf" event={"ID":"173d0d44-527b-47d0-99e2-3ffb1058ba31","Type":"ContainerStarted","Data":"50373308b04700064ba6862b2704bdf424a54b16bd68fe236e2520c5c39a3f3d"}
Sep 14 07:49:51.652050 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.651866    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf" event={"ID":"173d0d44-527b-47d0-99e2-3ffb1058ba31","Type":"ContainerStarted","Data":"ab8585da76aad4a9ca479ddebb5aca21c7a4b2b940bdae22b1e5c7be4020d265"}
Sep 14 07:49:51.652050 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.651909    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:49:51.653426 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.653402    2562 generic.go:356] "Generic (PLEG): container finished" podID="5324fdbc-a7c2-484c-b0a2-641594258899" containerID="1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554" exitCode=0
Sep 14 07:49:51.653531 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.653470    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5324fdbc-a7c2-484c-b0a2-641594258899","Type":"ContainerDied","Data":"1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554"}
Sep 14 07:49:51.653570 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.653561    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5324fdbc-a7c2-484c-b0a2-641594258899","Type":"ContainerStarted","Data":"1cb30023ddd21668fd0c7d11630a66458d2d57d4d1ba213598b28410c6e5ca13"}
Sep 14 07:49:51.676042 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.675986    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.103613775 podStartE2EDuration="6.675970098s" totalImagesPullingTime="3.561898109s" totalInitContainerRuntime="10.458214ms" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:51.674452705 +0000 UTC m=+94.130902697" watchObservedRunningTime="2026-09-14 07:49:51.675970098 +0000 UTC m=+94.132420091"
Sep 14 07:49:51.700040 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.699983    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf" podStartSLOduration=2.122233142 podStartE2EDuration="4.699966781s" totalImagesPullingTime="2.577733639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:51.699302719 +0000 UTC m=+94.155752713" watchObservedRunningTime="2026-09-14 07:49:51.699966781 +0000 UTC m=+94.156416773"
Sep 14 07:49:51.747728 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:51.747643    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:54.130707 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:54.130654    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:54.186772 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:54.186718    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:49:55.678080 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:55.678040    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5324fdbc-a7c2-484c-b0a2-641594258899","Type":"ContainerStarted","Data":"9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d"}
Sep 14 07:49:55.678574 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:55.678103    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5324fdbc-a7c2-484c-b0a2-641594258899","Type":"ContainerStarted","Data":"b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9"}
Sep 14 07:49:55.678574 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:55.678128    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5324fdbc-a7c2-484c-b0a2-641594258899","Type":"ContainerStarted","Data":"a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24"}
Sep 14 07:49:55.678574 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:55.678153    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5324fdbc-a7c2-484c-b0a2-641594258899","Type":"ContainerStarted","Data":"dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba"}
Sep 14 07:49:55.678574 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:55.678177    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5324fdbc-a7c2-484c-b0a2-641594258899","Type":"ContainerStarted","Data":"95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2"}
Sep 14 07:49:55.678574 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:55.678206    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5324fdbc-a7c2-484c-b0a2-641594258899","Type":"ContainerStarted","Data":"3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5"}
Sep 14 07:49:55.732274 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:55.732218    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.471770213 podStartE2EDuration="5.732201339s" totalImagesPullingTime="3.244764903s" totalInitContainerRuntime="15.666223ms" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:55.729923412 +0000 UTC m=+98.186373405" watchObservedRunningTime="2026-09-14 07:49:55.732201339 +0000 UTC m=+98.188651331"
Sep 14 07:49:56.166323 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:56.166285    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:49:57.664613 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:49:57.664582    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5c7dc76f84-gdmhf"
Sep 14 07:50:00.553564 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:50:00.553530    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-v7dql"
Sep 14 07:50:01.442273 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:50:01.442246    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:02.999596 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:50:02.999531    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:03.318734 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:50:03.318653    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:06.521515 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:50:06.521451    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:06.579173 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:50:06.579135    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:06.637004 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:50:06.636973    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:07.492881 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:50:07.492845    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:07.551784 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:50:07.551745    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:09.736630 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:50:09.736597    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:09.836863 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:50:09.836826    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:09.892488 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:50:09.892438    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:11.773245 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:50:11.773216    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:11.874184 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:50:11.874120    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:13.059643 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:50:13.059601    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:17.038553 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:50:17.038488    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:17.094709 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:50:17.094676    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:50:51.166105 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:50:51.166067    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:51.185568 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:50:51.185537    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:50:51.897882 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:50:51.897852    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:00.745337 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:00.745296    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:00.806590 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:00.806557    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:05.095426 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:05.095383    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 14 07:51:05.095834 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:05.095682    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="alertmanager" containerID="cri-o://a0978b2f7e9812c44da51c5f7d91354a6728b45c448654ea6c16d9c036b8c6a7" gracePeriod=120
Sep 14 07:51:05.095834 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:05.095755    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="kube-rbac-proxy-metric" containerID="cri-o://b7754e19912481bb5a1dab30cd3bfcb1ede8fe257a8b516896dd0dcd54c9febf" gracePeriod=120
Sep 14 07:51:05.095834 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:05.095806    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="config-reloader" containerID="cri-o://5346e64ef93847abe21314a25960c99dbed97761eef03d1f0b8e8da72cf020fe" gracePeriod=120
Sep 14 07:51:05.096007 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:05.095819    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="prom-label-proxy" containerID="cri-o://8ee39965614a4883e85b2c5b45c3620a804213b89418c3ed8c29d9cd9ee97319" gracePeriod=120
Sep 14 07:51:05.096007 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:05.095777    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="kube-rbac-proxy" containerID="cri-o://342c00dbb22ddffd7c5c13b9fc21d11857d9e2c8436611c7a7a595026a259b8b" gracePeriod=120
Sep 14 07:51:05.096007 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:05.095755    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="kube-rbac-proxy-web" containerID="cri-o://c5fe061eb85cd309b7d54af7ce14c1512d7987939afc3000ee3fbbb3729db9cb" gracePeriod=120
Sep 14 07:51:05.937733 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:05.937698    2562 generic.go:356] "Generic (PLEG): container finished" podID="5c023ea1-de79-46d0-963e-583ef335e625" containerID="8ee39965614a4883e85b2c5b45c3620a804213b89418c3ed8c29d9cd9ee97319" exitCode=0
Sep 14 07:51:05.937733 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:05.937724    2562 generic.go:356] "Generic (PLEG): container finished" podID="5c023ea1-de79-46d0-963e-583ef335e625" containerID="342c00dbb22ddffd7c5c13b9fc21d11857d9e2c8436611c7a7a595026a259b8b" exitCode=0
Sep 14 07:51:05.937733 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:05.937731    2562 generic.go:356] "Generic (PLEG): container finished" podID="5c023ea1-de79-46d0-963e-583ef335e625" containerID="5346e64ef93847abe21314a25960c99dbed97761eef03d1f0b8e8da72cf020fe" exitCode=0
Sep 14 07:51:05.937733 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:05.937736    2562 generic.go:356] "Generic (PLEG): container finished" podID="5c023ea1-de79-46d0-963e-583ef335e625" containerID="a0978b2f7e9812c44da51c5f7d91354a6728b45c448654ea6c16d9c036b8c6a7" exitCode=0
Sep 14 07:51:05.938010 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:05.937767    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c023ea1-de79-46d0-963e-583ef335e625","Type":"ContainerDied","Data":"8ee39965614a4883e85b2c5b45c3620a804213b89418c3ed8c29d9cd9ee97319"}
Sep 14 07:51:05.938010 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:05.937830    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c023ea1-de79-46d0-963e-583ef335e625","Type":"ContainerDied","Data":"342c00dbb22ddffd7c5c13b9fc21d11857d9e2c8436611c7a7a595026a259b8b"}
Sep 14 07:51:05.938010 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:05.937856    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c023ea1-de79-46d0-963e-583ef335e625","Type":"ContainerDied","Data":"5346e64ef93847abe21314a25960c99dbed97761eef03d1f0b8e8da72cf020fe"}
Sep 14 07:51:05.938010 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:05.937882    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c023ea1-de79-46d0-963e-583ef335e625","Type":"ContainerDied","Data":"a0978b2f7e9812c44da51c5f7d91354a6728b45c448654ea6c16d9c036b8c6a7"}
Sep 14 07:51:06.339630 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.339606    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:06.430232 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.430203    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c023ea1-de79-46d0-963e-583ef335e625-metrics-client-ca\") pod \"5c023ea1-de79-46d0-963e-583ef335e625\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") "
Sep 14 07:51:06.430232 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.430235    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-cluster-tls-config\") pod \"5c023ea1-de79-46d0-963e-583ef335e625\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") "
Sep 14 07:51:06.430433 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.430259    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c023ea1-de79-46d0-963e-583ef335e625-config-out\") pod \"5c023ea1-de79-46d0-963e-583ef335e625\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") "
Sep 14 07:51:06.430433 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.430293    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-kube-rbac-proxy-web\") pod \"5c023ea1-de79-46d0-963e-583ef335e625\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") "
Sep 14 07:51:06.430569 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.430479    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5c023ea1-de79-46d0-963e-583ef335e625-alertmanager-main-db\") pod \"5c023ea1-de79-46d0-963e-583ef335e625\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") "
Sep 14 07:51:06.430569 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.430543    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-web-config\") pod \"5c023ea1-de79-46d0-963e-583ef335e625\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") "
Sep 14 07:51:06.430671 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.430595    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-kube-rbac-proxy-metric\") pod \"5c023ea1-de79-46d0-963e-583ef335e625\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") "
Sep 14 07:51:06.430671 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.430621    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c023ea1-de79-46d0-963e-583ef335e625-alertmanager-trusted-ca-bundle\") pod \"5c023ea1-de79-46d0-963e-583ef335e625\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") "
Sep 14 07:51:06.430773 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.430650    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c023ea1-de79-46d0-963e-583ef335e625-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "5c023ea1-de79-46d0-963e-583ef335e625" (UID: "5c023ea1-de79-46d0-963e-583ef335e625"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:51:06.430773 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.430668    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cb6xf\" (UniqueName: \"kubernetes.io/projected/5c023ea1-de79-46d0-963e-583ef335e625-kube-api-access-cb6xf\") pod \"5c023ea1-de79-46d0-963e-583ef335e625\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") "
Sep 14 07:51:06.430874 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.430790    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c023ea1-de79-46d0-963e-583ef335e625-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "5c023ea1-de79-46d0-963e-583ef335e625" (UID: "5c023ea1-de79-46d0-963e-583ef335e625"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:51:06.430874 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.430816    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-kube-rbac-proxy\") pod \"5c023ea1-de79-46d0-963e-583ef335e625\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") "
Sep 14 07:51:06.430874 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.430857    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-config-volume\") pod \"5c023ea1-de79-46d0-963e-583ef335e625\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") "
Sep 14 07:51:06.431029 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.430899    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c023ea1-de79-46d0-963e-583ef335e625-tls-assets\") pod \"5c023ea1-de79-46d0-963e-583ef335e625\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") "
Sep 14 07:51:06.431029 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.430954    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-main-tls\") pod \"5c023ea1-de79-46d0-963e-583ef335e625\" (UID: \"5c023ea1-de79-46d0-963e-583ef335e625\") "
Sep 14 07:51:06.431524 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.431198    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c023ea1-de79-46d0-963e-583ef335e625-metrics-client-ca\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:06.431524 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.431219    2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5c023ea1-de79-46d0-963e-583ef335e625-alertmanager-main-db\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:06.432093 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.432063    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c023ea1-de79-46d0-963e-583ef335e625-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "5c023ea1-de79-46d0-963e-583ef335e625" (UID: "5c023ea1-de79-46d0-963e-583ef335e625"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:51:06.433416 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.433380    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c023ea1-de79-46d0-963e-583ef335e625-config-out" (OuterVolumeSpecName: "config-out") pod "5c023ea1-de79-46d0-963e-583ef335e625" (UID: "5c023ea1-de79-46d0-963e-583ef335e625"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:51:06.434009 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.433983    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c023ea1-de79-46d0-963e-583ef335e625-kube-api-access-cb6xf" (OuterVolumeSpecName: "kube-api-access-cb6xf") pod "5c023ea1-de79-46d0-963e-583ef335e625" (UID: "5c023ea1-de79-46d0-963e-583ef335e625"). InnerVolumeSpecName "kube-api-access-cb6xf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:51:06.434655 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.434573    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "5c023ea1-de79-46d0-963e-583ef335e625" (UID: "5c023ea1-de79-46d0-963e-583ef335e625"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:51:06.434655 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.434618    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "5c023ea1-de79-46d0-963e-583ef335e625" (UID: "5c023ea1-de79-46d0-963e-583ef335e625"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:51:06.434806 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.434667    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-config-volume" (OuterVolumeSpecName: "config-volume") pod "5c023ea1-de79-46d0-963e-583ef335e625" (UID: "5c023ea1-de79-46d0-963e-583ef335e625"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:51:06.434806 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.434703    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c023ea1-de79-46d0-963e-583ef335e625-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5c023ea1-de79-46d0-963e-583ef335e625" (UID: "5c023ea1-de79-46d0-963e-583ef335e625"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:51:06.435059 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.435024    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "5c023ea1-de79-46d0-963e-583ef335e625" (UID: "5c023ea1-de79-46d0-963e-583ef335e625"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:51:06.436277 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.436255    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "5c023ea1-de79-46d0-963e-583ef335e625" (UID: "5c023ea1-de79-46d0-963e-583ef335e625"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:51:06.438224 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.438189    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "5c023ea1-de79-46d0-963e-583ef335e625" (UID: "5c023ea1-de79-46d0-963e-583ef335e625"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:51:06.444560 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.444489    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-web-config" (OuterVolumeSpecName: "web-config") pod "5c023ea1-de79-46d0-963e-583ef335e625" (UID: "5c023ea1-de79-46d0-963e-583ef335e625"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:51:06.532047 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.532016    2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-main-tls\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:06.532047 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.532043    2562 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-cluster-tls-config\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:06.532047 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.532056    2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c023ea1-de79-46d0-963e-583ef335e625-config-out\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:06.532248 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.532066    2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:06.532248 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.532077    2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-web-config\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:06.532248 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.532085    2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:06.532248 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.532093    2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c023ea1-de79-46d0-963e-583ef335e625-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:06.532248 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.532102    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cb6xf\" (UniqueName: \"kubernetes.io/projected/5c023ea1-de79-46d0-963e-583ef335e625-kube-api-access-cb6xf\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:06.532248 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.532110    2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:06.532248 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.532118    2562 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5c023ea1-de79-46d0-963e-583ef335e625-config-volume\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:06.532248 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.532126    2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c023ea1-de79-46d0-963e-583ef335e625-tls-assets\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:06.945182 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.945147    2562 generic.go:356] "Generic (PLEG): container finished" podID="5c023ea1-de79-46d0-963e-583ef335e625" containerID="b7754e19912481bb5a1dab30cd3bfcb1ede8fe257a8b516896dd0dcd54c9febf" exitCode=0
Sep 14 07:51:06.945182 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.945173    2562 generic.go:356] "Generic (PLEG): container finished" podID="5c023ea1-de79-46d0-963e-583ef335e625" containerID="c5fe061eb85cd309b7d54af7ce14c1512d7987939afc3000ee3fbbb3729db9cb" exitCode=0
Sep 14 07:51:06.945353 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.945229    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:06.945353 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.945222    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c023ea1-de79-46d0-963e-583ef335e625","Type":"ContainerDied","Data":"b7754e19912481bb5a1dab30cd3bfcb1ede8fe257a8b516896dd0dcd54c9febf"}
Sep 14 07:51:06.945438 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.945388    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c023ea1-de79-46d0-963e-583ef335e625","Type":"ContainerDied","Data":"c5fe061eb85cd309b7d54af7ce14c1512d7987939afc3000ee3fbbb3729db9cb"}
Sep 14 07:51:06.945472 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.945436    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5c023ea1-de79-46d0-963e-583ef335e625","Type":"ContainerDied","Data":"e55d790504d85aa5f04750c9ef7510792700877d7369b0f57a2b295663c730a1"}
Sep 14 07:51:06.945472 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.945449    2562 scope.go:117] "RemoveContainer" containerID="8ee39965614a4883e85b2c5b45c3620a804213b89418c3ed8c29d9cd9ee97319"
Sep 14 07:51:06.953654 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.953635    2562 scope.go:117] "RemoveContainer" containerID="b7754e19912481bb5a1dab30cd3bfcb1ede8fe257a8b516896dd0dcd54c9febf"
Sep 14 07:51:06.960767 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.960742    2562 scope.go:117] "RemoveContainer" containerID="342c00dbb22ddffd7c5c13b9fc21d11857d9e2c8436611c7a7a595026a259b8b"
Sep 14 07:51:06.967379 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.967357    2562 scope.go:117] "RemoveContainer" containerID="c5fe061eb85cd309b7d54af7ce14c1512d7987939afc3000ee3fbbb3729db9cb"
Sep 14 07:51:06.971636 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.971613    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 14 07:51:06.974533 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.974495    2562 scope.go:117] "RemoveContainer" containerID="5346e64ef93847abe21314a25960c99dbed97761eef03d1f0b8e8da72cf020fe"
Sep 14 07:51:06.980724 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.980696    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 14 07:51:06.984670 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.983708    2562 scope.go:117] "RemoveContainer" containerID="a0978b2f7e9812c44da51c5f7d91354a6728b45c448654ea6c16d9c036b8c6a7"
Sep 14 07:51:06.991064 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.991042    2562 scope.go:117] "RemoveContainer" containerID="96550f091cb8f8a634783ff540bab4443b00e5f3d46a3af9f634beba6cd4a34c"
Sep 14 07:51:06.998049 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.998030    2562 scope.go:117] "RemoveContainer" containerID="8ee39965614a4883e85b2c5b45c3620a804213b89418c3ed8c29d9cd9ee97319"
Sep 14 07:51:06.998309 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:51:06.998289    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ee39965614a4883e85b2c5b45c3620a804213b89418c3ed8c29d9cd9ee97319\": container with ID starting with 8ee39965614a4883e85b2c5b45c3620a804213b89418c3ed8c29d9cd9ee97319 not found: ID does not exist" containerID="8ee39965614a4883e85b2c5b45c3620a804213b89418c3ed8c29d9cd9ee97319"
Sep 14 07:51:06.998348 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.998317    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ee39965614a4883e85b2c5b45c3620a804213b89418c3ed8c29d9cd9ee97319"} err="failed to get container status \"8ee39965614a4883e85b2c5b45c3620a804213b89418c3ed8c29d9cd9ee97319\": rpc error: code = NotFound desc = could not find container \"8ee39965614a4883e85b2c5b45c3620a804213b89418c3ed8c29d9cd9ee97319\": container with ID starting with 8ee39965614a4883e85b2c5b45c3620a804213b89418c3ed8c29d9cd9ee97319 not found: ID does not exist"
Sep 14 07:51:06.998348 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.998347    2562 scope.go:117] "RemoveContainer" containerID="b7754e19912481bb5a1dab30cd3bfcb1ede8fe257a8b516896dd0dcd54c9febf"
Sep 14 07:51:06.998664 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:51:06.998639    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7754e19912481bb5a1dab30cd3bfcb1ede8fe257a8b516896dd0dcd54c9febf\": container with ID starting with b7754e19912481bb5a1dab30cd3bfcb1ede8fe257a8b516896dd0dcd54c9febf not found: ID does not exist" containerID="b7754e19912481bb5a1dab30cd3bfcb1ede8fe257a8b516896dd0dcd54c9febf"
Sep 14 07:51:06.998719 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.998670    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7754e19912481bb5a1dab30cd3bfcb1ede8fe257a8b516896dd0dcd54c9febf"} err="failed to get container status \"b7754e19912481bb5a1dab30cd3bfcb1ede8fe257a8b516896dd0dcd54c9febf\": rpc error: code = NotFound desc = could not find container \"b7754e19912481bb5a1dab30cd3bfcb1ede8fe257a8b516896dd0dcd54c9febf\": container with ID starting with b7754e19912481bb5a1dab30cd3bfcb1ede8fe257a8b516896dd0dcd54c9febf not found: ID does not exist"
Sep 14 07:51:06.998719 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.998686    2562 scope.go:117] "RemoveContainer" containerID="342c00dbb22ddffd7c5c13b9fc21d11857d9e2c8436611c7a7a595026a259b8b"
Sep 14 07:51:06.998909 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:51:06.998893    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"342c00dbb22ddffd7c5c13b9fc21d11857d9e2c8436611c7a7a595026a259b8b\": container with ID starting with 342c00dbb22ddffd7c5c13b9fc21d11857d9e2c8436611c7a7a595026a259b8b not found: ID does not exist" containerID="342c00dbb22ddffd7c5c13b9fc21d11857d9e2c8436611c7a7a595026a259b8b"
Sep 14 07:51:06.998994 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.998912    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"342c00dbb22ddffd7c5c13b9fc21d11857d9e2c8436611c7a7a595026a259b8b"} err="failed to get container status \"342c00dbb22ddffd7c5c13b9fc21d11857d9e2c8436611c7a7a595026a259b8b\": rpc error: code = NotFound desc = could not find container \"342c00dbb22ddffd7c5c13b9fc21d11857d9e2c8436611c7a7a595026a259b8b\": container with ID starting with 342c00dbb22ddffd7c5c13b9fc21d11857d9e2c8436611c7a7a595026a259b8b not found: ID does not exist"
Sep 14 07:51:06.998994 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.998926    2562 scope.go:117] "RemoveContainer" containerID="c5fe061eb85cd309b7d54af7ce14c1512d7987939afc3000ee3fbbb3729db9cb"
Sep 14 07:51:06.999135 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:51:06.999122    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5fe061eb85cd309b7d54af7ce14c1512d7987939afc3000ee3fbbb3729db9cb\": container with ID starting with c5fe061eb85cd309b7d54af7ce14c1512d7987939afc3000ee3fbbb3729db9cb not found: ID does not exist" containerID="c5fe061eb85cd309b7d54af7ce14c1512d7987939afc3000ee3fbbb3729db9cb"
Sep 14 07:51:06.999170 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.999137    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5fe061eb85cd309b7d54af7ce14c1512d7987939afc3000ee3fbbb3729db9cb"} err="failed to get container status \"c5fe061eb85cd309b7d54af7ce14c1512d7987939afc3000ee3fbbb3729db9cb\": rpc error: code = NotFound desc = could not find container \"c5fe061eb85cd309b7d54af7ce14c1512d7987939afc3000ee3fbbb3729db9cb\": container with ID starting with c5fe061eb85cd309b7d54af7ce14c1512d7987939afc3000ee3fbbb3729db9cb not found: ID does not exist"
Sep 14 07:51:06.999170 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.999147    2562 scope.go:117] "RemoveContainer" containerID="5346e64ef93847abe21314a25960c99dbed97761eef03d1f0b8e8da72cf020fe"
Sep 14 07:51:06.999365 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:51:06.999351    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5346e64ef93847abe21314a25960c99dbed97761eef03d1f0b8e8da72cf020fe\": container with ID starting with 5346e64ef93847abe21314a25960c99dbed97761eef03d1f0b8e8da72cf020fe not found: ID does not exist" containerID="5346e64ef93847abe21314a25960c99dbed97761eef03d1f0b8e8da72cf020fe"
Sep 14 07:51:06.999399 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.999367    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5346e64ef93847abe21314a25960c99dbed97761eef03d1f0b8e8da72cf020fe"} err="failed to get container status \"5346e64ef93847abe21314a25960c99dbed97761eef03d1f0b8e8da72cf020fe\": rpc error: code = NotFound desc = could not find container \"5346e64ef93847abe21314a25960c99dbed97761eef03d1f0b8e8da72cf020fe\": container with ID starting with 5346e64ef93847abe21314a25960c99dbed97761eef03d1f0b8e8da72cf020fe not found: ID does not exist"
Sep 14 07:51:06.999399 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.999379    2562 scope.go:117] "RemoveContainer" containerID="a0978b2f7e9812c44da51c5f7d91354a6728b45c448654ea6c16d9c036b8c6a7"
Sep 14 07:51:06.999635 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:51:06.999615    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0978b2f7e9812c44da51c5f7d91354a6728b45c448654ea6c16d9c036b8c6a7\": container with ID starting with a0978b2f7e9812c44da51c5f7d91354a6728b45c448654ea6c16d9c036b8c6a7 not found: ID does not exist" containerID="a0978b2f7e9812c44da51c5f7d91354a6728b45c448654ea6c16d9c036b8c6a7"
Sep 14 07:51:06.999684 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.999640    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0978b2f7e9812c44da51c5f7d91354a6728b45c448654ea6c16d9c036b8c6a7"} err="failed to get container status \"a0978b2f7e9812c44da51c5f7d91354a6728b45c448654ea6c16d9c036b8c6a7\": rpc error: code = NotFound desc = could not find container \"a0978b2f7e9812c44da51c5f7d91354a6728b45c448654ea6c16d9c036b8c6a7\": container with ID starting with a0978b2f7e9812c44da51c5f7d91354a6728b45c448654ea6c16d9c036b8c6a7 not found: ID does not exist"
Sep 14 07:51:06.999684 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.999653    2562 scope.go:117] "RemoveContainer" containerID="96550f091cb8f8a634783ff540bab4443b00e5f3d46a3af9f634beba6cd4a34c"
Sep 14 07:51:06.999873 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:51:06.999853    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96550f091cb8f8a634783ff540bab4443b00e5f3d46a3af9f634beba6cd4a34c\": container with ID starting with 96550f091cb8f8a634783ff540bab4443b00e5f3d46a3af9f634beba6cd4a34c not found: ID does not exist" containerID="96550f091cb8f8a634783ff540bab4443b00e5f3d46a3af9f634beba6cd4a34c"
Sep 14 07:51:06.999932 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.999884    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96550f091cb8f8a634783ff540bab4443b00e5f3d46a3af9f634beba6cd4a34c"} err="failed to get container status \"96550f091cb8f8a634783ff540bab4443b00e5f3d46a3af9f634beba6cd4a34c\": rpc error: code = NotFound desc = could not find container \"96550f091cb8f8a634783ff540bab4443b00e5f3d46a3af9f634beba6cd4a34c\": container with ID starting with 96550f091cb8f8a634783ff540bab4443b00e5f3d46a3af9f634beba6cd4a34c not found: ID does not exist"
Sep 14 07:51:06.999932 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:06.999910    2562 scope.go:117] "RemoveContainer" containerID="8ee39965614a4883e85b2c5b45c3620a804213b89418c3ed8c29d9cd9ee97319"
Sep 14 07:51:07.000126 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:07.000110    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ee39965614a4883e85b2c5b45c3620a804213b89418c3ed8c29d9cd9ee97319"} err="failed to get container status \"8ee39965614a4883e85b2c5b45c3620a804213b89418c3ed8c29d9cd9ee97319\": rpc error: code = NotFound desc = could not find container \"8ee39965614a4883e85b2c5b45c3620a804213b89418c3ed8c29d9cd9ee97319\": container with ID starting with 8ee39965614a4883e85b2c5b45c3620a804213b89418c3ed8c29d9cd9ee97319 not found: ID does not exist"
Sep 14 07:51:07.000173 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:07.000127    2562 scope.go:117] "RemoveContainer" containerID="b7754e19912481bb5a1dab30cd3bfcb1ede8fe257a8b516896dd0dcd54c9febf"
Sep 14 07:51:07.000312 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:07.000293    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7754e19912481bb5a1dab30cd3bfcb1ede8fe257a8b516896dd0dcd54c9febf"} err="failed to get container status \"b7754e19912481bb5a1dab30cd3bfcb1ede8fe257a8b516896dd0dcd54c9febf\": rpc error: code = NotFound desc = could not find container \"b7754e19912481bb5a1dab30cd3bfcb1ede8fe257a8b516896dd0dcd54c9febf\": container with ID starting with b7754e19912481bb5a1dab30cd3bfcb1ede8fe257a8b516896dd0dcd54c9febf not found: ID does not exist"
Sep 14 07:51:07.000376 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:07.000315    2562 scope.go:117] "RemoveContainer" containerID="342c00dbb22ddffd7c5c13b9fc21d11857d9e2c8436611c7a7a595026a259b8b"
Sep 14 07:51:07.000540 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:07.000525    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"342c00dbb22ddffd7c5c13b9fc21d11857d9e2c8436611c7a7a595026a259b8b"} err="failed to get container status \"342c00dbb22ddffd7c5c13b9fc21d11857d9e2c8436611c7a7a595026a259b8b\": rpc error: code = NotFound desc = could not find container \"342c00dbb22ddffd7c5c13b9fc21d11857d9e2c8436611c7a7a595026a259b8b\": container with ID starting with 342c00dbb22ddffd7c5c13b9fc21d11857d9e2c8436611c7a7a595026a259b8b not found: ID does not exist"
Sep 14 07:51:07.000590 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:07.000540    2562 scope.go:117] "RemoveContainer" containerID="c5fe061eb85cd309b7d54af7ce14c1512d7987939afc3000ee3fbbb3729db9cb"
Sep 14 07:51:07.000751 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:07.000731    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5fe061eb85cd309b7d54af7ce14c1512d7987939afc3000ee3fbbb3729db9cb"} err="failed to get container status \"c5fe061eb85cd309b7d54af7ce14c1512d7987939afc3000ee3fbbb3729db9cb\": rpc error: code = NotFound desc = could not find container \"c5fe061eb85cd309b7d54af7ce14c1512d7987939afc3000ee3fbbb3729db9cb\": container with ID starting with c5fe061eb85cd309b7d54af7ce14c1512d7987939afc3000ee3fbbb3729db9cb not found: ID does not exist"
Sep 14 07:51:07.000822 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:07.000752    2562 scope.go:117] "RemoveContainer" containerID="5346e64ef93847abe21314a25960c99dbed97761eef03d1f0b8e8da72cf020fe"
Sep 14 07:51:07.000954 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:07.000938    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5346e64ef93847abe21314a25960c99dbed97761eef03d1f0b8e8da72cf020fe"} err="failed to get container status \"5346e64ef93847abe21314a25960c99dbed97761eef03d1f0b8e8da72cf020fe\": rpc error: code = NotFound desc = could not find container \"5346e64ef93847abe21314a25960c99dbed97761eef03d1f0b8e8da72cf020fe\": container with ID starting with 5346e64ef93847abe21314a25960c99dbed97761eef03d1f0b8e8da72cf020fe not found: ID does not exist"
Sep 14 07:51:07.000996 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:07.000954    2562 scope.go:117] "RemoveContainer" containerID="a0978b2f7e9812c44da51c5f7d91354a6728b45c448654ea6c16d9c036b8c6a7"
Sep 14 07:51:07.001144 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:07.001125    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0978b2f7e9812c44da51c5f7d91354a6728b45c448654ea6c16d9c036b8c6a7"} err="failed to get container status \"a0978b2f7e9812c44da51c5f7d91354a6728b45c448654ea6c16d9c036b8c6a7\": rpc error: code = NotFound desc = could not find container \"a0978b2f7e9812c44da51c5f7d91354a6728b45c448654ea6c16d9c036b8c6a7\": container with ID starting with a0978b2f7e9812c44da51c5f7d91354a6728b45c448654ea6c16d9c036b8c6a7 not found: ID does not exist"
Sep 14 07:51:07.001209 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:07.001145    2562 scope.go:117] "RemoveContainer" containerID="96550f091cb8f8a634783ff540bab4443b00e5f3d46a3af9f634beba6cd4a34c"
Sep 14 07:51:07.001332 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:07.001317    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96550f091cb8f8a634783ff540bab4443b00e5f3d46a3af9f634beba6cd4a34c"} err="failed to get container status \"96550f091cb8f8a634783ff540bab4443b00e5f3d46a3af9f634beba6cd4a34c\": rpc error: code = NotFound desc = could not find container \"96550f091cb8f8a634783ff540bab4443b00e5f3d46a3af9f634beba6cd4a34c\": container with ID starting with 96550f091cb8f8a634783ff540bab4443b00e5f3d46a3af9f634beba6cd4a34c not found: ID does not exist"
Sep 14 07:51:08.111012 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:08.110978    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c023ea1-de79-46d0-963e-583ef335e625" path="/var/lib/kubelet/pods/5c023ea1-de79-46d0-963e-583ef335e625/volumes"
Sep 14 07:51:09.362258 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.362220    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 07:51:09.362793 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.362611    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="prometheus" containerID="cri-o://3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5" gracePeriod=600
Sep 14 07:51:09.362793 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.362651    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="kube-rbac-proxy-thanos" containerID="cri-o://9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d" gracePeriod=600
Sep 14 07:51:09.362793 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.362659    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="kube-rbac-proxy-web" containerID="cri-o://a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24" gracePeriod=600
Sep 14 07:51:09.362793 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.362724    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="config-reloader" containerID="cri-o://95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2" gracePeriod=600
Sep 14 07:51:09.362793 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.362703    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="kube-rbac-proxy" containerID="cri-o://b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9" gracePeriod=600
Sep 14 07:51:09.363061 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.362682    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="thanos-sidecar" containerID="cri-o://dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba" gracePeriod=600
Sep 14 07:51:09.615624 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.615568    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:09.756352 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.756322    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5324fdbc-a7c2-484c-b0a2-641594258899-tls-assets\") pod \"5324fdbc-a7c2-484c-b0a2-641594258899\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") "
Sep 14 07:51:09.756571 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.756368    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-grpc-tls\") pod \"5324fdbc-a7c2-484c-b0a2-641594258899\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") "
Sep 14 07:51:09.756571 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.756396    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-db\") pod \"5324fdbc-a7c2-484c-b0a2-641594258899\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") "
Sep 14 07:51:09.756571 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.756438    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-configmap-kubelet-serving-ca-bundle\") pod \"5324fdbc-a7c2-484c-b0a2-641594258899\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") "
Sep 14 07:51:09.756571 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.756463    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-rulefiles-2\") pod \"5324fdbc-a7c2-484c-b0a2-641594258899\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") "
Sep 14 07:51:09.756571 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.756495    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"5324fdbc-a7c2-484c-b0a2-641594258899\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") "
Sep 14 07:51:09.756571 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.756548    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-kube-rbac-proxy\") pod \"5324fdbc-a7c2-484c-b0a2-641594258899\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") "
Sep 14 07:51:09.756571 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.756573    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-configmap-serving-certs-ca-bundle\") pod \"5324fdbc-a7c2-484c-b0a2-641594258899\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") "
Sep 14 07:51:09.756929 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.756604    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-prometheus-k8s-tls\") pod \"5324fdbc-a7c2-484c-b0a2-641594258899\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") "
Sep 14 07:51:09.756929 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.756631    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-configmap-metrics-client-ca\") pod \"5324fdbc-a7c2-484c-b0a2-641594258899\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") "
Sep 14 07:51:09.756929 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.756666    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-web-config\") pod \"5324fdbc-a7c2-484c-b0a2-641594258899\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") "
Sep 14 07:51:09.756929 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.756691    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-trusted-ca-bundle\") pod \"5324fdbc-a7c2-484c-b0a2-641594258899\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") "
Sep 14 07:51:09.756929 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.756723    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"5324fdbc-a7c2-484c-b0a2-641594258899\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") "
Sep 14 07:51:09.756929 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.756772    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-config\") pod \"5324fdbc-a7c2-484c-b0a2-641594258899\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") "
Sep 14 07:51:09.756929 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.756805    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-metrics-client-certs\") pod \"5324fdbc-a7c2-484c-b0a2-641594258899\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") "
Sep 14 07:51:09.756929 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.756839    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5324fdbc-a7c2-484c-b0a2-641594258899-config-out\") pod \"5324fdbc-a7c2-484c-b0a2-641594258899\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") "
Sep 14 07:51:09.756929 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.756863    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-rulefiles-0\") pod \"5324fdbc-a7c2-484c-b0a2-641594258899\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") "
Sep 14 07:51:09.756929 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.756887    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-thanos-prometheus-http-client-file\") pod \"5324fdbc-a7c2-484c-b0a2-641594258899\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") "
Sep 14 07:51:09.756929 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.756911    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znwtx\" (UniqueName: \"kubernetes.io/projected/5324fdbc-a7c2-484c-b0a2-641594258899-kube-api-access-znwtx\") pod \"5324fdbc-a7c2-484c-b0a2-641594258899\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") "
Sep 14 07:51:09.756929 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.756937    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-rulefiles-1\") pod \"5324fdbc-a7c2-484c-b0a2-641594258899\" (UID: \"5324fdbc-a7c2-484c-b0a2-641594258899\") "
Sep 14 07:51:09.758523 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.756828    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "5324fdbc-a7c2-484c-b0a2-641594258899" (UID: "5324fdbc-a7c2-484c-b0a2-641594258899"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:51:09.758523 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.756896    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "5324fdbc-a7c2-484c-b0a2-641594258899" (UID: "5324fdbc-a7c2-484c-b0a2-641594258899"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:51:09.758523 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.757451    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "5324fdbc-a7c2-484c-b0a2-641594258899" (UID: "5324fdbc-a7c2-484c-b0a2-641594258899"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:51:09.758523 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.757920    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "5324fdbc-a7c2-484c-b0a2-641594258899" (UID: "5324fdbc-a7c2-484c-b0a2-641594258899"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:51:09.758815 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.758622    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "5324fdbc-a7c2-484c-b0a2-641594258899" (UID: "5324fdbc-a7c2-484c-b0a2-641594258899"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:51:09.758815 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.758630    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "5324fdbc-a7c2-484c-b0a2-641594258899" (UID: "5324fdbc-a7c2-484c-b0a2-641594258899"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:51:09.758925 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.758832    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "5324fdbc-a7c2-484c-b0a2-641594258899" (UID: "5324fdbc-a7c2-484c-b0a2-641594258899"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:51:09.760106 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.759368    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5324fdbc-a7c2-484c-b0a2-641594258899-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5324fdbc-a7c2-484c-b0a2-641594258899" (UID: "5324fdbc-a7c2-484c-b0a2-641594258899"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:51:09.760106 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.759613    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "5324fdbc-a7c2-484c-b0a2-641594258899" (UID: "5324fdbc-a7c2-484c-b0a2-641594258899"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:51:09.760106 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.759724    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5324fdbc-a7c2-484c-b0a2-641594258899-config-out" (OuterVolumeSpecName: "config-out") pod "5324fdbc-a7c2-484c-b0a2-641594258899" (UID: "5324fdbc-a7c2-484c-b0a2-641594258899"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:51:09.760106 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.759841    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "5324fdbc-a7c2-484c-b0a2-641594258899" (UID: "5324fdbc-a7c2-484c-b0a2-641594258899"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:51:09.760106 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.759973    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "5324fdbc-a7c2-484c-b0a2-641594258899" (UID: "5324fdbc-a7c2-484c-b0a2-641594258899"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:51:09.760106 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.760081    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "5324fdbc-a7c2-484c-b0a2-641594258899" (UID: "5324fdbc-a7c2-484c-b0a2-641594258899"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:51:09.760926 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.760882    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "5324fdbc-a7c2-484c-b0a2-641594258899" (UID: "5324fdbc-a7c2-484c-b0a2-641594258899"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:51:09.761020 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.760963    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "5324fdbc-a7c2-484c-b0a2-641594258899" (UID: "5324fdbc-a7c2-484c-b0a2-641594258899"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:51:09.761767 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.761740    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "5324fdbc-a7c2-484c-b0a2-641594258899" (UID: "5324fdbc-a7c2-484c-b0a2-641594258899"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:51:09.761852 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.761794    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5324fdbc-a7c2-484c-b0a2-641594258899-kube-api-access-znwtx" (OuterVolumeSpecName: "kube-api-access-znwtx") pod "5324fdbc-a7c2-484c-b0a2-641594258899" (UID: "5324fdbc-a7c2-484c-b0a2-641594258899"). InnerVolumeSpecName "kube-api-access-znwtx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:51:09.761852 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.761813    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "5324fdbc-a7c2-484c-b0a2-641594258899" (UID: "5324fdbc-a7c2-484c-b0a2-641594258899"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:51:09.762873 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.762851    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-config" (OuterVolumeSpecName: "config") pod "5324fdbc-a7c2-484c-b0a2-641594258899" (UID: "5324fdbc-a7c2-484c-b0a2-641594258899"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:51:09.768617 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.768597    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-web-config" (OuterVolumeSpecName: "web-config") pod "5324fdbc-a7c2-484c-b0a2-641594258899" (UID: "5324fdbc-a7c2-484c-b0a2-641594258899"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:51:09.858432 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.858398    2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-web-config\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:09.858432 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.858430    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:09.858432 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.858442    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:09.858652 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.858451    2562 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-config\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:09.858652 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.858460    2562 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-metrics-client-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:09.858652 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.858469    2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5324fdbc-a7c2-484c-b0a2-641594258899-config-out\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:09.858652 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.858477    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:09.858652 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.858485    2562 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:09.858652 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.858494    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znwtx\" (UniqueName: \"kubernetes.io/projected/5324fdbc-a7c2-484c-b0a2-641594258899-kube-api-access-znwtx\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:09.858652 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.858520    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:09.858652 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.858528    2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5324fdbc-a7c2-484c-b0a2-641594258899-tls-assets\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:09.858652 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.858536    2562 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-grpc-tls\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:09.858652 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.858543    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-db\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:09.858652 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.858551    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:09.858652 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.858559    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:09.858652 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.858568    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:09.858652 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.858576    2562 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-kube-rbac-proxy\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:09.858652 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.858584    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:09.858652 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.858593    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5324fdbc-a7c2-484c-b0a2-641594258899-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:09.858652 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.858601    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5324fdbc-a7c2-484c-b0a2-641594258899-configmap-metrics-client-ca\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:51:09.959110 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.959079    2562 generic.go:356] "Generic (PLEG): container finished" podID="5324fdbc-a7c2-484c-b0a2-641594258899" containerID="9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d" exitCode=0
Sep 14 07:51:09.959110 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.959104    2562 generic.go:356] "Generic (PLEG): container finished" podID="5324fdbc-a7c2-484c-b0a2-641594258899" containerID="b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9" exitCode=0
Sep 14 07:51:09.959110 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.959110    2562 generic.go:356] "Generic (PLEG): container finished" podID="5324fdbc-a7c2-484c-b0a2-641594258899" containerID="a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24" exitCode=0
Sep 14 07:51:09.959110 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.959117    2562 generic.go:356] "Generic (PLEG): container finished" podID="5324fdbc-a7c2-484c-b0a2-641594258899" containerID="dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba" exitCode=0
Sep 14 07:51:09.959413 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.959123    2562 generic.go:356] "Generic (PLEG): container finished" podID="5324fdbc-a7c2-484c-b0a2-641594258899" containerID="95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2" exitCode=0
Sep 14 07:51:09.959413 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.959130    2562 generic.go:356] "Generic (PLEG): container finished" podID="5324fdbc-a7c2-484c-b0a2-641594258899" containerID="3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5" exitCode=0
Sep 14 07:51:09.959413 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.959163    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:09.959413 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.959165    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5324fdbc-a7c2-484c-b0a2-641594258899","Type":"ContainerDied","Data":"9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d"}
Sep 14 07:51:09.959413 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.959239    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5324fdbc-a7c2-484c-b0a2-641594258899","Type":"ContainerDied","Data":"b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9"}
Sep 14 07:51:09.959413 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.959274    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5324fdbc-a7c2-484c-b0a2-641594258899","Type":"ContainerDied","Data":"a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24"}
Sep 14 07:51:09.959413 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.959305    2562 scope.go:117] "RemoveContainer" containerID="9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d"
Sep 14 07:51:09.959413 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.959303    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5324fdbc-a7c2-484c-b0a2-641594258899","Type":"ContainerDied","Data":"dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba"}
Sep 14 07:51:09.959413 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.959391    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5324fdbc-a7c2-484c-b0a2-641594258899","Type":"ContainerDied","Data":"95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2"}
Sep 14 07:51:09.959828 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.959433    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5324fdbc-a7c2-484c-b0a2-641594258899","Type":"ContainerDied","Data":"3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5"}
Sep 14 07:51:09.959828 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.959468    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5324fdbc-a7c2-484c-b0a2-641594258899","Type":"ContainerDied","Data":"1cb30023ddd21668fd0c7d11630a66458d2d57d4d1ba213598b28410c6e5ca13"}
Sep 14 07:51:09.967109 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.966970    2562 scope.go:117] "RemoveContainer" containerID="b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9"
Sep 14 07:51:09.973929 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.973908    2562 scope.go:117] "RemoveContainer" containerID="a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24"
Sep 14 07:51:09.980525 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.980487    2562 scope.go:117] "RemoveContainer" containerID="dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba"
Sep 14 07:51:09.983915 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.983891    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 07:51:09.989474 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.989452    2562 scope.go:117] "RemoveContainer" containerID="95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2"
Sep 14 07:51:09.990457 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.990437    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 07:51:09.996236 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:09.996218    2562 scope.go:117] "RemoveContainer" containerID="3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5"
Sep 14 07:51:10.003040 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.003022    2562 scope.go:117] "RemoveContainer" containerID="1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554"
Sep 14 07:51:10.009609 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.009580    2562 scope.go:117] "RemoveContainer" containerID="9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d"
Sep 14 07:51:10.009846 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:51:10.009829    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d\": container with ID starting with 9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d not found: ID does not exist" containerID="9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d"
Sep 14 07:51:10.009915 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.009855    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d"} err="failed to get container status \"9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d\": rpc error: code = NotFound desc = could not find container \"9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d\": container with ID starting with 9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d not found: ID does not exist"
Sep 14 07:51:10.009915 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.009874    2562 scope.go:117] "RemoveContainer" containerID="b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9"
Sep 14 07:51:10.010117 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:51:10.010093    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9\": container with ID starting with b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9 not found: ID does not exist" containerID="b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9"
Sep 14 07:51:10.010185 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.010119    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9"} err="failed to get container status \"b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9\": rpc error: code = NotFound desc = could not find container \"b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9\": container with ID starting with b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9 not found: ID does not exist"
Sep 14 07:51:10.010185 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.010141    2562 scope.go:117] "RemoveContainer" containerID="a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24"
Sep 14 07:51:10.010403 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:51:10.010384    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24\": container with ID starting with a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24 not found: ID does not exist" containerID="a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24"
Sep 14 07:51:10.010441 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.010407    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24"} err="failed to get container status \"a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24\": rpc error: code = NotFound desc = could not find container \"a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24\": container with ID starting with a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24 not found: ID does not exist"
Sep 14 07:51:10.010441 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.010421    2562 scope.go:117] "RemoveContainer" containerID="dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba"
Sep 14 07:51:10.010665 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:51:10.010648    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba\": container with ID starting with dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba not found: ID does not exist" containerID="dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba"
Sep 14 07:51:10.010734 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.010672    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba"} err="failed to get container status \"dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba\": rpc error: code = NotFound desc = could not find container \"dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba\": container with ID starting with dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba not found: ID does not exist"
Sep 14 07:51:10.010734 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.010690    2562 scope.go:117] "RemoveContainer" containerID="95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2"
Sep 14 07:51:10.010919 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:51:10.010901    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2\": container with ID starting with 95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2 not found: ID does not exist" containerID="95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2"
Sep 14 07:51:10.010964 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.010925    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2"} err="failed to get container status \"95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2\": rpc error: code = NotFound desc = could not find container \"95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2\": container with ID starting with 95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2 not found: ID does not exist"
Sep 14 07:51:10.010964 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.010939    2562 scope.go:117] "RemoveContainer" containerID="3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5"
Sep 14 07:51:10.011181 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:51:10.011160    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5\": container with ID starting with 3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5 not found: ID does not exist" containerID="3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5"
Sep 14 07:51:10.011225 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.011187    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5"} err="failed to get container status \"3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5\": rpc error: code = NotFound desc = could not find container \"3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5\": container with ID starting with 3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5 not found: ID does not exist"
Sep 14 07:51:10.011225 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.011203    2562 scope.go:117] "RemoveContainer" containerID="1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554"
Sep 14 07:51:10.011431 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:51:10.011414    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554\": container with ID starting with 1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554 not found: ID does not exist" containerID="1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554"
Sep 14 07:51:10.011487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.011438    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554"} err="failed to get container status \"1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554\": rpc error: code = NotFound desc = could not find container \"1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554\": container with ID starting with 1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554 not found: ID does not exist"
Sep 14 07:51:10.011487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.011457    2562 scope.go:117] "RemoveContainer" containerID="9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d"
Sep 14 07:51:10.011732 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.011716    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d"} err="failed to get container status \"9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d\": rpc error: code = NotFound desc = could not find container \"9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d\": container with ID starting with 9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d not found: ID does not exist"
Sep 14 07:51:10.011781 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.011731    2562 scope.go:117] "RemoveContainer" containerID="b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9"
Sep 14 07:51:10.011969 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.011952    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9"} err="failed to get container status \"b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9\": rpc error: code = NotFound desc = could not find container \"b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9\": container with ID starting with b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9 not found: ID does not exist"
Sep 14 07:51:10.012012 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.011970    2562 scope.go:117] "RemoveContainer" containerID="a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24"
Sep 14 07:51:10.012188 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.012171    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24"} err="failed to get container status \"a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24\": rpc error: code = NotFound desc = could not find container \"a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24\": container with ID starting with a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24 not found: ID does not exist"
Sep 14 07:51:10.012253 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.012190    2562 scope.go:117] "RemoveContainer" containerID="dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba"
Sep 14 07:51:10.012437 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.012422    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba"} err="failed to get container status \"dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba\": rpc error: code = NotFound desc = could not find container \"dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba\": container with ID starting with dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba not found: ID does not exist"
Sep 14 07:51:10.012486 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.012437    2562 scope.go:117] "RemoveContainer" containerID="95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2"
Sep 14 07:51:10.012653 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.012637    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2"} err="failed to get container status \"95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2\": rpc error: code = NotFound desc = could not find container \"95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2\": container with ID starting with 95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2 not found: ID does not exist"
Sep 14 07:51:10.012697 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.012654    2562 scope.go:117] "RemoveContainer" containerID="3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5"
Sep 14 07:51:10.012846 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.012826    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5"} err="failed to get container status \"3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5\": rpc error: code = NotFound desc = could not find container \"3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5\": container with ID starting with 3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5 not found: ID does not exist"
Sep 14 07:51:10.012912 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.012847    2562 scope.go:117] "RemoveContainer" containerID="1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554"
Sep 14 07:51:10.013105 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.013087    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554"} err="failed to get container status \"1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554\": rpc error: code = NotFound desc = could not find container \"1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554\": container with ID starting with 1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554 not found: ID does not exist"
Sep 14 07:51:10.013144 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.013105    2562 scope.go:117] "RemoveContainer" containerID="9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d"
Sep 14 07:51:10.013294 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.013279    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d"} err="failed to get container status \"9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d\": rpc error: code = NotFound desc = could not find container \"9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d\": container with ID starting with 9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d not found: ID does not exist"
Sep 14 07:51:10.013342 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.013293    2562 scope.go:117] "RemoveContainer" containerID="b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9"
Sep 14 07:51:10.013469 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.013454    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9"} err="failed to get container status \"b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9\": rpc error: code = NotFound desc = could not find container \"b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9\": container with ID starting with b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9 not found: ID does not exist"
Sep 14 07:51:10.013469 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.013468    2562 scope.go:117] "RemoveContainer" containerID="a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24"
Sep 14 07:51:10.013709 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.013692    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24"} err="failed to get container status \"a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24\": rpc error: code = NotFound desc = could not find container \"a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24\": container with ID starting with a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24 not found: ID does not exist"
Sep 14 07:51:10.013748 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.013709    2562 scope.go:117] "RemoveContainer" containerID="dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba"
Sep 14 07:51:10.013957 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.013937    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba"} err="failed to get container status \"dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba\": rpc error: code = NotFound desc = could not find container \"dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba\": container with ID starting with dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba not found: ID does not exist"
Sep 14 07:51:10.014017 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.013957    2562 scope.go:117] "RemoveContainer" containerID="95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2"
Sep 14 07:51:10.014199 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.014182    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2"} err="failed to get container status \"95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2\": rpc error: code = NotFound desc = could not find container \"95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2\": container with ID starting with 95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2 not found: ID does not exist"
Sep 14 07:51:10.014247 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.014200    2562 scope.go:117] "RemoveContainer" containerID="3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5"
Sep 14 07:51:10.014404 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.014387    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5"} err="failed to get container status \"3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5\": rpc error: code = NotFound desc = could not find container \"3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5\": container with ID starting with 3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5 not found: ID does not exist"
Sep 14 07:51:10.014442 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.014405    2562 scope.go:117] "RemoveContainer" containerID="1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554"
Sep 14 07:51:10.014612 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.014596    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554"} err="failed to get container status \"1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554\": rpc error: code = NotFound desc = could not find container \"1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554\": container with ID starting with 1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554 not found: ID does not exist"
Sep 14 07:51:10.014650 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.014612    2562 scope.go:117] "RemoveContainer" containerID="9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d"
Sep 14 07:51:10.014818 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.014802    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d"} err="failed to get container status \"9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d\": rpc error: code = NotFound desc = could not find container \"9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d\": container with ID starting with 9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d not found: ID does not exist"
Sep 14 07:51:10.014866 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.014819    2562 scope.go:117] "RemoveContainer" containerID="b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9"
Sep 14 07:51:10.015052 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.015035    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9"} err="failed to get container status \"b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9\": rpc error: code = NotFound desc = could not find container \"b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9\": container with ID starting with b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9 not found: ID does not exist"
Sep 14 07:51:10.015102 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.015052    2562 scope.go:117] "RemoveContainer" containerID="a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24"
Sep 14 07:51:10.015302 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.015282    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24"} err="failed to get container status \"a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24\": rpc error: code = NotFound desc = could not find container \"a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24\": container with ID starting with a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24 not found: ID does not exist"
Sep 14 07:51:10.015370 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.015304    2562 scope.go:117] "RemoveContainer" containerID="dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba"
Sep 14 07:51:10.015522 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.015489    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba"} err="failed to get container status \"dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba\": rpc error: code = NotFound desc = could not find container \"dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba\": container with ID starting with dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba not found: ID does not exist"
Sep 14 07:51:10.015522 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.015521    2562 scope.go:117] "RemoveContainer" containerID="95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2"
Sep 14 07:51:10.015733 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.015716    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2"} err="failed to get container status \"95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2\": rpc error: code = NotFound desc = could not find container \"95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2\": container with ID starting with 95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2 not found: ID does not exist"
Sep 14 07:51:10.015775 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.015733    2562 scope.go:117] "RemoveContainer" containerID="3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5"
Sep 14 07:51:10.015955 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.015937    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5"} err="failed to get container status \"3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5\": rpc error: code = NotFound desc = could not find container \"3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5\": container with ID starting with 3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5 not found: ID does not exist"
Sep 14 07:51:10.016002 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.015956    2562 scope.go:117] "RemoveContainer" containerID="1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554"
Sep 14 07:51:10.016177 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.016157    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554"} err="failed to get container status \"1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554\": rpc error: code = NotFound desc = could not find container \"1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554\": container with ID starting with 1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554 not found: ID does not exist"
Sep 14 07:51:10.016235 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.016177    2562 scope.go:117] "RemoveContainer" containerID="9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d"
Sep 14 07:51:10.016382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.016366    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d"} err="failed to get container status \"9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d\": rpc error: code = NotFound desc = could not find container \"9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d\": container with ID starting with 9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d not found: ID does not exist"
Sep 14 07:51:10.016418 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.016383    2562 scope.go:117] "RemoveContainer" containerID="b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9"
Sep 14 07:51:10.016598 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.016583    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9"} err="failed to get container status \"b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9\": rpc error: code = NotFound desc = could not find container \"b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9\": container with ID starting with b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9 not found: ID does not exist"
Sep 14 07:51:10.016598 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.016597    2562 scope.go:117] "RemoveContainer" containerID="a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24"
Sep 14 07:51:10.016793 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.016777    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24"} err="failed to get container status \"a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24\": rpc error: code = NotFound desc = could not find container \"a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24\": container with ID starting with a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24 not found: ID does not exist"
Sep 14 07:51:10.016837 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.016793    2562 scope.go:117] "RemoveContainer" containerID="dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba"
Sep 14 07:51:10.017003 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.016988    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba"} err="failed to get container status \"dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba\": rpc error: code = NotFound desc = could not find container \"dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba\": container with ID starting with dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba not found: ID does not exist"
Sep 14 07:51:10.017053 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.017004    2562 scope.go:117] "RemoveContainer" containerID="95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2"
Sep 14 07:51:10.017215 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.017200    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2"} err="failed to get container status \"95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2\": rpc error: code = NotFound desc = could not find container \"95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2\": container with ID starting with 95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2 not found: ID does not exist"
Sep 14 07:51:10.017261 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.017215    2562 scope.go:117] "RemoveContainer" containerID="3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5"
Sep 14 07:51:10.017536 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.017484    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5"} err="failed to get container status \"3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5\": rpc error: code = NotFound desc = could not find container \"3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5\": container with ID starting with 3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5 not found: ID does not exist"
Sep 14 07:51:10.017536 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.017535    2562 scope.go:117] "RemoveContainer" containerID="1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554"
Sep 14 07:51:10.017764 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.017746    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554"} err="failed to get container status \"1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554\": rpc error: code = NotFound desc = could not find container \"1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554\": container with ID starting with 1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554 not found: ID does not exist"
Sep 14 07:51:10.017824 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.017764    2562 scope.go:117] "RemoveContainer" containerID="9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d"
Sep 14 07:51:10.018010 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.017984    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d"} err="failed to get container status \"9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d\": rpc error: code = NotFound desc = could not find container \"9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d\": container with ID starting with 9d2c27be68c923140429eb102f034e17fd2a3f8fb0125d36d36e1a89e39ef32d not found: ID does not exist"
Sep 14 07:51:10.018010 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.018009    2562 scope.go:117] "RemoveContainer" containerID="b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9"
Sep 14 07:51:10.018356 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.018325    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9"} err="failed to get container status \"b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9\": rpc error: code = NotFound desc = could not find container \"b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9\": container with ID starting with b3be320832da20374d8344d25731732e048a2fb3d4c0b010464d9654e7f8c7e9 not found: ID does not exist"
Sep 14 07:51:10.018356 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.018356    2562 scope.go:117] "RemoveContainer" containerID="a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24"
Sep 14 07:51:10.018756 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.018712    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24"} err="failed to get container status \"a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24\": rpc error: code = NotFound desc = could not find container \"a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24\": container with ID starting with a5536ce5e4121b7ef8c570d4910dec56c6ec7a01fe46d14397e7888611b4fb24 not found: ID does not exist"
Sep 14 07:51:10.018756 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.018736    2562 scope.go:117] "RemoveContainer" containerID="dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba"
Sep 14 07:51:10.019030 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.019012    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba"} err="failed to get container status \"dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba\": rpc error: code = NotFound desc = could not find container \"dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba\": container with ID starting with dd60179ce5895db00f7e30cf6932395f5087ec6a2412dfaa3c4ac7077915beba not found: ID does not exist"
Sep 14 07:51:10.019109 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.019031    2562 scope.go:117] "RemoveContainer" containerID="95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2"
Sep 14 07:51:10.019798 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.019767    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2"} err="failed to get container status \"95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2\": rpc error: code = NotFound desc = could not find container \"95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2\": container with ID starting with 95f0916495af66798da86cfdf3d6f725ea0599389fd3b11b0f5db267a69bc1b2 not found: ID does not exist"
Sep 14 07:51:10.019798 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.019798    2562 scope.go:117] "RemoveContainer" containerID="3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5"
Sep 14 07:51:10.019930 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.019866    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 07:51:10.020080 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020061    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5"} err="failed to get container status \"3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5\": rpc error: code = NotFound desc = could not find container \"3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5\": container with ID starting with 3f0ba4342ff6a20dcac1b111fcbce35d9ec468f7272e59d40a4e7ea674cacda5 not found: ID does not exist"
Sep 14 07:51:10.020129 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020080    2562 scope.go:117] "RemoveContainer" containerID="1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554"
Sep 14 07:51:10.020129 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020120    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="init-config-reloader"
Sep 14 07:51:10.020200 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020132    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="init-config-reloader"
Sep 14 07:51:10.020200 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020141    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="kube-rbac-proxy-web"
Sep 14 07:51:10.020200 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020148    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="kube-rbac-proxy-web"
Sep 14 07:51:10.020200 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020156    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="alertmanager"
Sep 14 07:51:10.020200 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020162    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="alertmanager"
Sep 14 07:51:10.020200 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020172    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="thanos-sidecar"
Sep 14 07:51:10.020200 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020180    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="thanos-sidecar"
Sep 14 07:51:10.020200 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020190    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="kube-rbac-proxy"
Sep 14 07:51:10.020200 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020197    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="kube-rbac-proxy"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020206    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="kube-rbac-proxy-metric"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020211    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="kube-rbac-proxy-metric"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020219    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="kube-rbac-proxy"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020224    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="kube-rbac-proxy"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020233    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="prometheus"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020241    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="prometheus"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020249    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="config-reloader"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020257    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="config-reloader"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020266    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="kube-rbac-proxy-thanos"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020272    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="kube-rbac-proxy-thanos"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020279    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="kube-rbac-proxy-web"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020284    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="kube-rbac-proxy-web"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020291    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="init-config-reloader"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020297    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="init-config-reloader"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020304    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="prom-label-proxy"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020309    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="prom-label-proxy"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020314    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="config-reloader"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020318    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="config-reloader"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020289    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554"} err="failed to get container status \"1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554\": rpc error: code = NotFound desc = could not find container \"1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554\": container with ID starting with 1129e9001b0fb5fe1999b86e5612c83a9710f6d6a06b780ed63e8cf5da62b554 not found: ID does not exist"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020372    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="kube-rbac-proxy"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020381    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="config-reloader"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020386    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="thanos-sidecar"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020392    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="kube-rbac-proxy-thanos"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020399    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="kube-rbac-proxy"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020404    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="prom-label-proxy"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020409    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="kube-rbac-proxy-web"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020415    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="config-reloader"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020420    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="kube-rbac-proxy-metric"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020427    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="kube-rbac-proxy-web"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020431    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c023ea1-de79-46d0-963e-583ef335e625" containerName="alertmanager"
Sep 14 07:51:10.020458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.020437    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" containerName="prometheus"
Sep 14 07:51:10.025183 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.025167    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.027525 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.027489    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 14 07:51:10.027633 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.027617    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 14 07:51:10.028311 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.028295    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 14 07:51:10.028839 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.028820    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 14 07:51:10.028901 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.028878    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 14 07:51:10.028989 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.028973    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 14 07:51:10.029109 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.029090    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5vtms\""
Sep 14 07:51:10.029163 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.029112    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1ns2lg6g237s5\""
Sep 14 07:51:10.029163 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.029099    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 14 07:51:10.031772 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.030379    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 14 07:51:10.031772 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.030718    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 14 07:51:10.031772 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.030751    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 14 07:51:10.031969 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.031918    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 14 07:51:10.033846 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.032276    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 14 07:51:10.033846 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.032327    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 14 07:51:10.036189 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.036167    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 14 07:51:10.037717 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.037699    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 14 07:51:10.047450 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.047429    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 07:51:10.111919 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.111887    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5324fdbc-a7c2-484c-b0a2-641594258899" path="/var/lib/kubelet/pods/5324fdbc-a7c2-484c-b0a2-641594258899/volumes"
Sep 14 07:51:10.160336 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.160300    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.160453 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.160340    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.160453 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.160362    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.160453 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.160378    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dlqp\" (UniqueName: \"kubernetes.io/projected/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-kube-api-access-4dlqp\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.160453 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.160448    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.160610 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.160463    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.160610 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.160540    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.160610 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.160589    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.160769 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.160612    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.160769 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.160633    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.160769 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.160658    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.160769 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.160687    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-web-config\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.160769 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.160707    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-config\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.160769 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.160721    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-config-out\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.160769 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.160748    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.161062 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.160802    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.161062 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.160847    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.161062 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.160870    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.161062 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.160901    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.161062 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.160935    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.262398 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.262284    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.262398 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.262319    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dlqp\" (UniqueName: \"kubernetes.io/projected/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-kube-api-access-4dlqp\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.262398 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.262356    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.262398 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.262382    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.262679 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.262411    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.262679 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.262440    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.262679 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.262465    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.262679 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.262531    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.262679 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.262559    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.262679 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.262588    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-web-config\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.262679 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.262622    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-config\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.262679 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.262644    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-config-out\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.262679 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.262673    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.263126 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.262701    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.263126 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.262735    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.263126 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.262759    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.263126 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.262769    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.263126 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.262794    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.263126 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.262831    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.263126 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.262863    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.263126 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.262886    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.263540 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.263316    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.263540 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.263519    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.264396 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.264080    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.265322 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.265295    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-config-out\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.265864 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.265714    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.265864 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.265731    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.266017 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.265864    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.266078 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.266021    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.266078 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.266039    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.266533 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.266224    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.266533 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.266452    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.266678 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.266548    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.266832 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.266811    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.266892 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.266845    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.267854 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.267833    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-web-config\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.268377 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.268353    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.268483 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.268464    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-config\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.268613 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.268596    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.273653 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.273633    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dlqp\" (UniqueName: \"kubernetes.io/projected/5d75856f-ee1d-42ac-acfc-b57ec2965a2b-kube-api-access-4dlqp\") pod \"prometheus-k8s-0\" (UID: \"5d75856f-ee1d-42ac-acfc-b57ec2965a2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.338967 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.338925    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:10.626491 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.626418    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 14 07:51:10.629695 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:51:10.629667    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d75856f_ee1d_42ac_acfc_b57ec2965a2b.slice/crio-05fbf812e8b379cfcbb0abbfc2ecda840e7a0c1b75363d6d84e1148c2c675b49 WatchSource:0}: Error finding container 05fbf812e8b379cfcbb0abbfc2ecda840e7a0c1b75363d6d84e1148c2c675b49: Status 404 returned error can't find the container with id 05fbf812e8b379cfcbb0abbfc2ecda840e7a0c1b75363d6d84e1148c2c675b49
Sep 14 07:51:10.966035 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.965998    2562 generic.go:356] "Generic (PLEG): container finished" podID="5d75856f-ee1d-42ac-acfc-b57ec2965a2b" containerID="34dc39e59e5b8c74813a172c1a755c836a51378b5e19fbf31ce81703f1696f24" exitCode=0
Sep 14 07:51:10.966232 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.966082    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d75856f-ee1d-42ac-acfc-b57ec2965a2b","Type":"ContainerDied","Data":"34dc39e59e5b8c74813a172c1a755c836a51378b5e19fbf31ce81703f1696f24"}
Sep 14 07:51:10.966232 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:10.966167    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d75856f-ee1d-42ac-acfc-b57ec2965a2b","Type":"ContainerStarted","Data":"05fbf812e8b379cfcbb0abbfc2ecda840e7a0c1b75363d6d84e1148c2c675b49"}
Sep 14 07:51:11.973880 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:11.973841    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d75856f-ee1d-42ac-acfc-b57ec2965a2b","Type":"ContainerStarted","Data":"8eeb6fb79f293c663cf7457121388e7a4911ddc094e049a15fec432adb4047d6"}
Sep 14 07:51:11.974333 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:11.973956    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d75856f-ee1d-42ac-acfc-b57ec2965a2b","Type":"ContainerStarted","Data":"5a6c69c3fee66654b504b071eb73fa55cfd1f3a4671119cb2262b147bded40f6"}
Sep 14 07:51:11.974333 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:11.974010    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d75856f-ee1d-42ac-acfc-b57ec2965a2b","Type":"ContainerStarted","Data":"d74659304d53fd86d62808b6b19761f2f162a616c0402837013e26d913f0149c"}
Sep 14 07:51:11.974333 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:11.974053    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d75856f-ee1d-42ac-acfc-b57ec2965a2b","Type":"ContainerStarted","Data":"c8ad167e8ef02005b021cafde96b7d12e7774f88f400153994bed5d1b952b326"}
Sep 14 07:51:11.974333 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:11.974090    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d75856f-ee1d-42ac-acfc-b57ec2965a2b","Type":"ContainerStarted","Data":"c002e968c68d060781c9d9e961d027c9e35a006977399fb8ddbaf788b1cbe86c"}
Sep 14 07:51:11.974333 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:11.974129    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5d75856f-ee1d-42ac-acfc-b57ec2965a2b","Type":"ContainerStarted","Data":"2b379d093f22272c4781cf4e32548ecdff1057f9d18e9200730600ab0948b4dc"}
Sep 14 07:51:12.008473 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:12.008414    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.989507815 podStartE2EDuration="3.00839526s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.887445ms" isStatefulPod=false podCreationTimestamp="2026-09-14 07:51:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:51:12.006824266 +0000 UTC m=+174.463274283" watchObservedRunningTime="2026-09-14 07:51:12.00839526 +0000 UTC m=+174.464845254"
Sep 14 07:51:12.392037 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:12.391955    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:12.448608 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:12.448576    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:12.749375 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:12.749332    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:12.814091 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:12.814057    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:15.339516 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:15.339464    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:51:18.013543 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:18.013513    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:18.114435 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:18.114392    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:18.177853 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:18.177766    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:22.880452 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:22.880414    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:22.973325 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:22.973288    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:23.067192 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:23.067156    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:28.001634 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:28.001603    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:28.056954 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:28.056921    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:51.513797 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:51.513752    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:51:51.572234 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:51:51.572198    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:04.439872 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:04.439830    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:04.537169 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:04.537128    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:10.339423 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:10.339381    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:52:10.354724 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:10.354697    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:52:11.204281 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:11.204254    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 14 07:52:12.536618 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:12.536585    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:30.886416 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:30.886370    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:32.251876 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:32.251836    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:34.491844 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:34.491807    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:34.558574 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:34.558535    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:35.921928 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:35.921889    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:36.020223 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:36.020190    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:36.496003 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:36.495970    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:36.610387 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:36.610350    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:38.134427 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:38.134398    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:38.334841 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:38.334800    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:41.563080 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:41.562994    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:41.621170 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:41.621138    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:44.725135 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:44.725102    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:44.778914 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:44.778887    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:46.555583 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:46.555544    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:49.161741 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:49.161700    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:49.263004 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:49.262972    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:52.353356 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:52.353319    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:52.430230 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:52.430193    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:55.706578 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:55.706530    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:55.806862 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:55.806829    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:52:57.833140 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:52:57.833105    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:12.304728 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:12.304698    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:12.404761 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:12.404725    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:12.999961 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:12.999925    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:13.102664 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.102633    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:13.210605 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.210571    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct"]
Sep 14 07:53:13.213684 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.213664    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct"
Sep 14 07:53:13.215991 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.215969    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 14 07:53:13.216234 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.216217    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 14 07:53:13.216304 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.216263    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-79xct\""
Sep 14 07:53:13.216304 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.216279    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 14 07:53:13.216304 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.216288    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 14 07:53:13.227382 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.227361    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct"]
Sep 14 07:53:13.321484 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.321397    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:13.326481 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.326458    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1962415f-77ed-420f-afa0-38b3947646ef-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c4c5b687-twcct\" (UID: \"1962415f-77ed-420f-afa0-38b3947646ef\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct"
Sep 14 07:53:13.326565 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.326490    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1962415f-77ed-420f-afa0-38b3947646ef-manifests\") pod \"opendatahub-operator-controller-manager-84c4c5b687-twcct\" (UID: \"1962415f-77ed-420f-afa0-38b3947646ef\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct"
Sep 14 07:53:13.326614 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.326560    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1962415f-77ed-420f-afa0-38b3947646ef-tmp\") pod \"opendatahub-operator-controller-manager-84c4c5b687-twcct\" (UID: \"1962415f-77ed-420f-afa0-38b3947646ef\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct"
Sep 14 07:53:13.326614 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.326602    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75mbh\" (UniqueName: \"kubernetes.io/projected/1962415f-77ed-420f-afa0-38b3947646ef-kube-api-access-75mbh\") pod \"opendatahub-operator-controller-manager-84c4c5b687-twcct\" (UID: \"1962415f-77ed-420f-afa0-38b3947646ef\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct"
Sep 14 07:53:13.326689 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.326650    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1962415f-77ed-420f-afa0-38b3947646ef-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c4c5b687-twcct\" (UID: \"1962415f-77ed-420f-afa0-38b3947646ef\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct"
Sep 14 07:53:13.421388 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.421353    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:13.427442 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.427419    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1962415f-77ed-420f-afa0-38b3947646ef-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c4c5b687-twcct\" (UID: \"1962415f-77ed-420f-afa0-38b3947646ef\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct"
Sep 14 07:53:13.427524 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.427466    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1962415f-77ed-420f-afa0-38b3947646ef-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c4c5b687-twcct\" (UID: \"1962415f-77ed-420f-afa0-38b3947646ef\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct"
Sep 14 07:53:13.427524 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.427493    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1962415f-77ed-420f-afa0-38b3947646ef-manifests\") pod \"opendatahub-operator-controller-manager-84c4c5b687-twcct\" (UID: \"1962415f-77ed-420f-afa0-38b3947646ef\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct"
Sep 14 07:53:13.427599 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.427549    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1962415f-77ed-420f-afa0-38b3947646ef-tmp\") pod \"opendatahub-operator-controller-manager-84c4c5b687-twcct\" (UID: \"1962415f-77ed-420f-afa0-38b3947646ef\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct"
Sep 14 07:53:13.427599 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.427580    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75mbh\" (UniqueName: \"kubernetes.io/projected/1962415f-77ed-420f-afa0-38b3947646ef-kube-api-access-75mbh\") pod \"opendatahub-operator-controller-manager-84c4c5b687-twcct\" (UID: \"1962415f-77ed-420f-afa0-38b3947646ef\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct"
Sep 14 07:53:13.428000 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.427979    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1962415f-77ed-420f-afa0-38b3947646ef-manifests\") pod \"opendatahub-operator-controller-manager-84c4c5b687-twcct\" (UID: \"1962415f-77ed-420f-afa0-38b3947646ef\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct"
Sep 14 07:53:13.429763 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.429731    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1962415f-77ed-420f-afa0-38b3947646ef-tmp\") pod \"opendatahub-operator-controller-manager-84c4c5b687-twcct\" (UID: \"1962415f-77ed-420f-afa0-38b3947646ef\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct"
Sep 14 07:53:13.429955 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.429936    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1962415f-77ed-420f-afa0-38b3947646ef-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c4c5b687-twcct\" (UID: \"1962415f-77ed-420f-afa0-38b3947646ef\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct"
Sep 14 07:53:13.430051 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.430032    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1962415f-77ed-420f-afa0-38b3947646ef-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c4c5b687-twcct\" (UID: \"1962415f-77ed-420f-afa0-38b3947646ef\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct"
Sep 14 07:53:13.436465 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.436442    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75mbh\" (UniqueName: \"kubernetes.io/projected/1962415f-77ed-420f-afa0-38b3947646ef-kube-api-access-75mbh\") pod \"opendatahub-operator-controller-manager-84c4c5b687-twcct\" (UID: \"1962415f-77ed-420f-afa0-38b3947646ef\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct"
Sep 14 07:53:13.523601 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.523550    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct"
Sep 14 07:53:13.548438 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.548407    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:13.629980 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.629948    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:13.820726 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.820689    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct"]
Sep 14 07:53:13.827085 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:53:13.827056    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1962415f_77ed_420f_afa0_38b3947646ef.slice/crio-87d87f2b1b0bd9532a7b5374f0c61d53d47f028e74c8feea7c8032975b9d610b WatchSource:0}: Error finding container 87d87f2b1b0bd9532a7b5374f0c61d53d47f028e74c8feea7c8032975b9d610b: Status 404 returned error can't find the container with id 87d87f2b1b0bd9532a7b5374f0c61d53d47f028e74c8feea7c8032975b9d610b
Sep 14 07:53:13.948884 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:13.948852    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:14.425766 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:14.425730    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct" event={"ID":"1962415f-77ed-420f-afa0-38b3947646ef","Type":"ContainerStarted","Data":"87d87f2b1b0bd9532a7b5374f0c61d53d47f028e74c8feea7c8032975b9d610b"}
Sep 14 07:53:17.440199 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:17.440157    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct" event={"ID":"1962415f-77ed-420f-afa0-38b3947646ef","Type":"ContainerStarted","Data":"5bdc1c645ecfb017dd443b48391a18db59a9f7a56ba2d86a4283535d5c697f5b"}
Sep 14 07:53:17.890899 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:17.890876    2562 kubelet.go:1648] "Image garbage collection succeeded"
Sep 14 07:53:18.445294 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:18.445253    2562 generic.go:356] "Generic (PLEG): container finished" podID="1962415f-77ed-420f-afa0-38b3947646ef" containerID="5bdc1c645ecfb017dd443b48391a18db59a9f7a56ba2d86a4283535d5c697f5b" exitCode=0
Sep 14 07:53:18.447678 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:18.445311    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct" event={"ID":"1962415f-77ed-420f-afa0-38b3947646ef","Type":"ContainerDied","Data":"5bdc1c645ecfb017dd443b48391a18db59a9f7a56ba2d86a4283535d5c697f5b"}
Sep 14 07:53:19.450968 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:19.450930    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct" event={"ID":"1962415f-77ed-420f-afa0-38b3947646ef","Type":"ContainerStarted","Data":"87c1bcf6d7838418f1be0609b8fb4cbc64919c07dd13584a5ff6491f8716c254"}
Sep 14 07:53:19.451391 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:19.451067    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct"
Sep 14 07:53:19.476201 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:19.476148    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct" podStartSLOduration=2.119705634 podStartE2EDuration="6.476132703s" totalImagesPullingTime="3.283610207s" totalInitContainerRuntime="1.072816862s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:53:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:53:19.474989601 +0000 UTC m=+301.931439612" watchObservedRunningTime="2026-09-14 07:53:19.476132703 +0000 UTC m=+301.932582696"
Sep 14 07:53:23.575448 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:23.575411    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:25.113491 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:25.113461    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:30.457642 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:30.457601    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-twcct"
Sep 14 07:53:32.317347 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:32.317309    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:32.486375 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:32.486342    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:32.730686 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:32.730641    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:33.051573 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:33.051473    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:34.175857 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:34.175811    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:34.238895 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:34.238861    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:35.388441 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:35.388403    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:39.005875 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.005834    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:39.106177 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.106128    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:39.285079 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.284992    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:39.320892 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.320859    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"]
Sep 14 07:53:39.335930 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.335897    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"]
Sep 14 07:53:39.336086 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.336026    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.336717 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.336693    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/101c9562-855a-4776-9df4-c40aaef6eb5a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.336858 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.336724    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/101c9562-855a-4776-9df4-c40aaef6eb5a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.336858 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.336761    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/101c9562-855a-4776-9df4-c40aaef6eb5a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.336955 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.336856    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/101c9562-855a-4776-9df4-c40aaef6eb5a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.336955 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.336906    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/101c9562-855a-4776-9df4-c40aaef6eb5a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.336955 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.336933    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/101c9562-855a-4776-9df4-c40aaef6eb5a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.337110 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.336962    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/101c9562-855a-4776-9df4-c40aaef6eb5a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.337110 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.336998    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/101c9562-855a-4776-9df4-c40aaef6eb5a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.337110 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.337090    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55ps4\" (UniqueName: \"kubernetes.io/projected/101c9562-855a-4776-9df4-c40aaef6eb5a-kube-api-access-55ps4\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.338488 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.338466    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 14 07:53:39.338646 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.338515    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 14 07:53:39.338763 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.338739    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 14 07:53:39.338848 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.338813    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-s4z8k\""
Sep 14 07:53:39.405335 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.405295    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:39.437652 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.437616    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/101c9562-855a-4776-9df4-c40aaef6eb5a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.437652 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.437652    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/101c9562-855a-4776-9df4-c40aaef6eb5a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.437836 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.437676    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/101c9562-855a-4776-9df4-c40aaef6eb5a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.437836 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.437699    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/101c9562-855a-4776-9df4-c40aaef6eb5a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.437836 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.437726    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/101c9562-855a-4776-9df4-c40aaef6eb5a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.437836 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.437748    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/101c9562-855a-4776-9df4-c40aaef6eb5a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.437836 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.437768    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/101c9562-855a-4776-9df4-c40aaef6eb5a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.437836 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.437789    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/101c9562-855a-4776-9df4-c40aaef6eb5a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.438159 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.437858    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55ps4\" (UniqueName: \"kubernetes.io/projected/101c9562-855a-4776-9df4-c40aaef6eb5a-kube-api-access-55ps4\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.438159 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.438077    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/101c9562-855a-4776-9df4-c40aaef6eb5a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.438159 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.438113    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/101c9562-855a-4776-9df4-c40aaef6eb5a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.438335 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.438188    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/101c9562-855a-4776-9df4-c40aaef6eb5a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.438432 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.438412    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/101c9562-855a-4776-9df4-c40aaef6eb5a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.438533 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.438491    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/101c9562-855a-4776-9df4-c40aaef6eb5a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.440226 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.440204    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/101c9562-855a-4776-9df4-c40aaef6eb5a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.440718 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.440701    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/101c9562-855a-4776-9df4-c40aaef6eb5a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.447074 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.447051    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/101c9562-855a-4776-9df4-c40aaef6eb5a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.447300 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.447280    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55ps4\" (UniqueName: \"kubernetes.io/projected/101c9562-855a-4776-9df4-c40aaef6eb5a-kube-api-access-55ps4\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf\" (UID: \"101c9562-855a-4776-9df4-c40aaef6eb5a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.600317 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.600238    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:39.647473 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.647438    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:39.923369 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.923181    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"]
Sep 14 07:53:39.925847 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:53:39.925819    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod101c9562_855a_4776_9df4_c40aaef6eb5a.slice/crio-3e2b4ddd870104f94a73ff9789a681edc6ac2ce4a6ec5ac450ec1b56a09137df WatchSource:0}: Error finding container 3e2b4ddd870104f94a73ff9789a681edc6ac2ce4a6ec5ac450ec1b56a09137df: Status 404 returned error can't find the container with id 3e2b4ddd870104f94a73ff9789a681edc6ac2ce4a6ec5ac450ec1b56a09137df
Sep 14 07:53:39.927911 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:39.927893    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 07:53:40.535375 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:40.535325    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf" event={"ID":"101c9562-855a-4776-9df4-c40aaef6eb5a","Type":"ContainerStarted","Data":"3e2b4ddd870104f94a73ff9789a681edc6ac2ce4a6ec5ac450ec1b56a09137df"}
Sep 14 07:53:42.455646 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:42.455610    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 07:53:42.455923 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:42.455681    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 07:53:42.455923 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:42.455707    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 07:53:42.547590 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:42.547542    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf" event={"ID":"101c9562-855a-4776-9df4-c40aaef6eb5a","Type":"ContainerStarted","Data":"010ff1370a000ff05bcfaf0dd8ca6d4da281690f9210fdd6239405f13d711a9e"}
Sep 14 07:53:42.572129 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:42.572068    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf" podStartSLOduration=1.044699385 podStartE2EDuration="3.572043725s" totalImagesPullingTime="2.52734434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:53:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:53:42.569803133 +0000 UTC m=+325.026253126" watchObservedRunningTime="2026-09-14 07:53:42.572043725 +0000 UTC m=+325.028493719"
Sep 14 07:53:42.648187 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:42.648146    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:43.258152 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:43.258116    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:43.354581 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:43.354546    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:43.455746 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:43.455707    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:43.597157 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:43.597066    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:43.648973 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:43.648937    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8mnfx"]
Sep 14 07:53:43.653959 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:43.653924    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-8mnfx"
Sep 14 07:53:43.657124 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:43.657096    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-d4989\""
Sep 14 07:53:43.657241 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:43.657096    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 14 07:53:43.658242 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:43.658222    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:43.660930 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:43.660887    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8mnfx"]
Sep 14 07:53:43.679608 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:43.679573    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:43.772563 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:43.772487    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/646ed64e-44c7-4bce-a9fb-0a8ff74e6774-tls-certs\") pod \"model-serving-api-54d89bd5fc-8mnfx\" (UID: \"646ed64e-44c7-4bce-a9fb-0a8ff74e6774\") " pod="opendatahub/model-serving-api-54d89bd5fc-8mnfx"
Sep 14 07:53:43.772743 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:43.772617    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8r7g\" (UniqueName: \"kubernetes.io/projected/646ed64e-44c7-4bce-a9fb-0a8ff74e6774-kube-api-access-z8r7g\") pod \"model-serving-api-54d89bd5fc-8mnfx\" (UID: \"646ed64e-44c7-4bce-a9fb-0a8ff74e6774\") " pod="opendatahub/model-serving-api-54d89bd5fc-8mnfx"
Sep 14 07:53:43.873669 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:43.873574    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/646ed64e-44c7-4bce-a9fb-0a8ff74e6774-tls-certs\") pod \"model-serving-api-54d89bd5fc-8mnfx\" (UID: \"646ed64e-44c7-4bce-a9fb-0a8ff74e6774\") " pod="opendatahub/model-serving-api-54d89bd5fc-8mnfx"
Sep 14 07:53:43.873669 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:43.873627    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8r7g\" (UniqueName: \"kubernetes.io/projected/646ed64e-44c7-4bce-a9fb-0a8ff74e6774-kube-api-access-z8r7g\") pod \"model-serving-api-54d89bd5fc-8mnfx\" (UID: \"646ed64e-44c7-4bce-a9fb-0a8ff74e6774\") " pod="opendatahub/model-serving-api-54d89bd5fc-8mnfx"
Sep 14 07:53:43.876036 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:43.876005    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/646ed64e-44c7-4bce-a9fb-0a8ff74e6774-tls-certs\") pod \"model-serving-api-54d89bd5fc-8mnfx\" (UID: \"646ed64e-44c7-4bce-a9fb-0a8ff74e6774\") " pod="opendatahub/model-serving-api-54d89bd5fc-8mnfx"
Sep 14 07:53:43.884051 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:43.884022    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8r7g\" (UniqueName: \"kubernetes.io/projected/646ed64e-44c7-4bce-a9fb-0a8ff74e6774-kube-api-access-z8r7g\") pod \"model-serving-api-54d89bd5fc-8mnfx\" (UID: \"646ed64e-44c7-4bce-a9fb-0a8ff74e6774\") " pod="opendatahub/model-serving-api-54d89bd5fc-8mnfx"
Sep 14 07:53:43.966200 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:43.966159    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-8mnfx"
Sep 14 07:53:44.242334 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:44.242301    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-8mnfx"]
Sep 14 07:53:44.245097 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:53:44.245067    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod646ed64e_44c7_4bce_a9fb_0a8ff74e6774.slice/crio-f71d266d18ea2699d2719611fd2ea26aeba106e24971edc8403dbe57964c31c4 WatchSource:0}: Error finding container f71d266d18ea2699d2719611fd2ea26aeba106e24971edc8403dbe57964c31c4: Status 404 returned error can't find the container with id f71d266d18ea2699d2719611fd2ea26aeba106e24971edc8403dbe57964c31c4
Sep 14 07:53:44.556985 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:44.556898    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-8mnfx" event={"ID":"646ed64e-44c7-4bce-a9fb-0a8ff74e6774","Type":"ContainerStarted","Data":"f71d266d18ea2699d2719611fd2ea26aeba106e24971edc8403dbe57964c31c4"}
Sep 14 07:53:44.557345 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:44.557087    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:44.558042 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:44.558022    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf"
Sep 14 07:53:47.570496 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:47.570459    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-8mnfx" event={"ID":"646ed64e-44c7-4bce-a9fb-0a8ff74e6774","Type":"ContainerStarted","Data":"4a5357b2293b9a78cb102c8a51d4352b5c3684edd307b7e01da0fe5b8456c967"}
Sep 14 07:53:47.570896 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:47.570548    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-8mnfx"
Sep 14 07:53:47.590122 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:47.590065    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-8mnfx" podStartSLOduration=2.238853865 podStartE2EDuration="4.590047724s" totalImagesPullingTime="2.351193859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:53:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:53:47.589054364 +0000 UTC m=+330.045504356" watchObservedRunningTime="2026-09-14 07:53:47.590047724 +0000 UTC m=+330.046497716"
Sep 14 07:53:50.075291 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.075253    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-784955c84b-v8hmn"]
Sep 14 07:53:50.078655 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.078639    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-784955c84b-v8hmn"
Sep 14 07:53:50.080660 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.080638    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 14 07:53:50.081022 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.081006    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-fcpdl\""
Sep 14 07:53:50.081316 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.081280    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 14 07:53:50.081316 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.081307    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:53:50.081479 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.081393    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 14 07:53:50.081479 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.081412    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 14 07:53:50.086831 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.086809    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-784955c84b-v8hmn"]
Sep 14 07:53:50.130174 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.130133    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/756e615b-8060-4e08-9d61-81588cb34016-metrics-cert\") pod \"lws-controller-manager-784955c84b-v8hmn\" (UID: \"756e615b-8060-4e08-9d61-81588cb34016\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-v8hmn"
Sep 14 07:53:50.130353 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.130214    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/756e615b-8060-4e08-9d61-81588cb34016-manager-config\") pod \"lws-controller-manager-784955c84b-v8hmn\" (UID: \"756e615b-8060-4e08-9d61-81588cb34016\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-v8hmn"
Sep 14 07:53:50.130353 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.130249    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjfm2\" (UniqueName: \"kubernetes.io/projected/756e615b-8060-4e08-9d61-81588cb34016-kube-api-access-pjfm2\") pod \"lws-controller-manager-784955c84b-v8hmn\" (UID: \"756e615b-8060-4e08-9d61-81588cb34016\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-v8hmn"
Sep 14 07:53:50.130353 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.130269    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/756e615b-8060-4e08-9d61-81588cb34016-cert\") pod \"lws-controller-manager-784955c84b-v8hmn\" (UID: \"756e615b-8060-4e08-9d61-81588cb34016\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-v8hmn"
Sep 14 07:53:50.230848 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.230810    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/756e615b-8060-4e08-9d61-81588cb34016-metrics-cert\") pod \"lws-controller-manager-784955c84b-v8hmn\" (UID: \"756e615b-8060-4e08-9d61-81588cb34016\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-v8hmn"
Sep 14 07:53:50.230848 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.230859    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/756e615b-8060-4e08-9d61-81588cb34016-manager-config\") pod \"lws-controller-manager-784955c84b-v8hmn\" (UID: \"756e615b-8060-4e08-9d61-81588cb34016\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-v8hmn"
Sep 14 07:53:50.231096 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.230882    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjfm2\" (UniqueName: \"kubernetes.io/projected/756e615b-8060-4e08-9d61-81588cb34016-kube-api-access-pjfm2\") pod \"lws-controller-manager-784955c84b-v8hmn\" (UID: \"756e615b-8060-4e08-9d61-81588cb34016\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-v8hmn"
Sep 14 07:53:50.231096 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.230899    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/756e615b-8060-4e08-9d61-81588cb34016-cert\") pod \"lws-controller-manager-784955c84b-v8hmn\" (UID: \"756e615b-8060-4e08-9d61-81588cb34016\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-v8hmn"
Sep 14 07:53:50.231569 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.231539    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/756e615b-8060-4e08-9d61-81588cb34016-manager-config\") pod \"lws-controller-manager-784955c84b-v8hmn\" (UID: \"756e615b-8060-4e08-9d61-81588cb34016\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-v8hmn"
Sep 14 07:53:50.233413 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.233382    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/756e615b-8060-4e08-9d61-81588cb34016-metrics-cert\") pod \"lws-controller-manager-784955c84b-v8hmn\" (UID: \"756e615b-8060-4e08-9d61-81588cb34016\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-v8hmn"
Sep 14 07:53:50.233537 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.233443    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/756e615b-8060-4e08-9d61-81588cb34016-cert\") pod \"lws-controller-manager-784955c84b-v8hmn\" (UID: \"756e615b-8060-4e08-9d61-81588cb34016\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-v8hmn"
Sep 14 07:53:50.240807 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.240777    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjfm2\" (UniqueName: \"kubernetes.io/projected/756e615b-8060-4e08-9d61-81588cb34016-kube-api-access-pjfm2\") pod \"lws-controller-manager-784955c84b-v8hmn\" (UID: \"756e615b-8060-4e08-9d61-81588cb34016\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-v8hmn"
Sep 14 07:53:50.388924 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.388834    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-784955c84b-v8hmn"
Sep 14 07:53:50.677273 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:50.677193    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-784955c84b-v8hmn"]
Sep 14 07:53:50.680469 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:53:50.680429    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod756e615b_8060_4e08_9d61_81588cb34016.slice/crio-84db1bd62b9e98ec9f4bece1a89894defc96f08c2ae52db4d48ed426046bbcc6 WatchSource:0}: Error finding container 84db1bd62b9e98ec9f4bece1a89894defc96f08c2ae52db4d48ed426046bbcc6: Status 404 returned error can't find the container with id 84db1bd62b9e98ec9f4bece1a89894defc96f08c2ae52db4d48ed426046bbcc6
Sep 14 07:53:51.588683 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:51.588645    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-784955c84b-v8hmn" event={"ID":"756e615b-8060-4e08-9d61-81588cb34016","Type":"ContainerStarted","Data":"84db1bd62b9e98ec9f4bece1a89894defc96f08c2ae52db4d48ed426046bbcc6"}
Sep 14 07:53:53.598756 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:53.598711    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-784955c84b-v8hmn" event={"ID":"756e615b-8060-4e08-9d61-81588cb34016","Type":"ContainerStarted","Data":"472229f8150b3f71bdc4230c87a3cb802aee563e953d42d04b90101cf29436d3"}
Sep 14 07:53:53.599160 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:53.598793    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-784955c84b-v8hmn"
Sep 14 07:53:53.636952 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:53.636904    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-784955c84b-v8hmn" podStartSLOduration=1.242901323 podStartE2EDuration="3.636883652s" totalImagesPullingTime="2.393982329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:53:53.636099367 +0000 UTC m=+336.092549360" watchObservedRunningTime="2026-09-14 07:53:53.636883652 +0000 UTC m=+336.093333646"
Sep 14 07:53:53.688632 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:53.688596    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:53.796698 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:53.796657    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hxw5j"]
Sep 14 07:53:53.801445 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:53.801413    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hxw5j"
Sep 14 07:53:53.804558 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:53.802281    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:53.806009 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:53.805986    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 14 07:53:53.808121 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:53.808096    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-qsk6q\""
Sep 14 07:53:53.815732 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:53.815707    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hxw5j"]
Sep 14 07:53:53.859606 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:53.859495    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmbnp\" (UniqueName: \"kubernetes.io/projected/db58eff4-ee8e-497f-a4e7-3e76297e5a7f-kube-api-access-vmbnp\") pod \"kserve-controller-manager-d6f4b65b8-hxw5j\" (UID: \"db58eff4-ee8e-497f-a4e7-3e76297e5a7f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hxw5j"
Sep 14 07:53:53.859766 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:53.859635    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db58eff4-ee8e-497f-a4e7-3e76297e5a7f-cert\") pod \"kserve-controller-manager-d6f4b65b8-hxw5j\" (UID: \"db58eff4-ee8e-497f-a4e7-3e76297e5a7f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hxw5j"
Sep 14 07:53:53.887214 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:53.887164    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:53.960197 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:53.960157    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmbnp\" (UniqueName: \"kubernetes.io/projected/db58eff4-ee8e-497f-a4e7-3e76297e5a7f-kube-api-access-vmbnp\") pod \"kserve-controller-manager-d6f4b65b8-hxw5j\" (UID: \"db58eff4-ee8e-497f-a4e7-3e76297e5a7f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hxw5j"
Sep 14 07:53:53.960335 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:53.960213    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db58eff4-ee8e-497f-a4e7-3e76297e5a7f-cert\") pod \"kserve-controller-manager-d6f4b65b8-hxw5j\" (UID: \"db58eff4-ee8e-497f-a4e7-3e76297e5a7f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hxw5j"
Sep 14 07:53:53.962615 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:53.962584    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db58eff4-ee8e-497f-a4e7-3e76297e5a7f-cert\") pod \"kserve-controller-manager-d6f4b65b8-hxw5j\" (UID: \"db58eff4-ee8e-497f-a4e7-3e76297e5a7f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hxw5j"
Sep 14 07:53:53.973582 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:53.973559    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmbnp\" (UniqueName: \"kubernetes.io/projected/db58eff4-ee8e-497f-a4e7-3e76297e5a7f-kube-api-access-vmbnp\") pod \"kserve-controller-manager-d6f4b65b8-hxw5j\" (UID: \"db58eff4-ee8e-497f-a4e7-3e76297e5a7f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-hxw5j"
Sep 14 07:53:53.990412 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:53.990376    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:54.092829 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:54.092791    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:54.117259 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:54.117182    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hxw5j"
Sep 14 07:53:54.170526 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:54.168331    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:53:54.407233 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:54.407158    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-hxw5j"]
Sep 14 07:53:54.410372 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:53:54.410346    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb58eff4_ee8e_497f_a4e7_3e76297e5a7f.slice/crio-fdd4692bd43dc8c1bd403177f781d9be8a8ebc1c4394b7e21750e85a15f75355 WatchSource:0}: Error finding container fdd4692bd43dc8c1bd403177f781d9be8a8ebc1c4394b7e21750e85a15f75355: Status 404 returned error can't find the container with id fdd4692bd43dc8c1bd403177f781d9be8a8ebc1c4394b7e21750e85a15f75355
Sep 14 07:53:54.604217 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:54.604178    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hxw5j" event={"ID":"db58eff4-ee8e-497f-a4e7-3e76297e5a7f","Type":"ContainerStarted","Data":"fdd4692bd43dc8c1bd403177f781d9be8a8ebc1c4394b7e21750e85a15f75355"}
Sep 14 07:53:57.621644 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:57.621559    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hxw5j" event={"ID":"db58eff4-ee8e-497f-a4e7-3e76297e5a7f","Type":"ContainerStarted","Data":"080d6b3ccc0662511513bae045f87d93482119a2ade91076e670bb3464fcf599"}
Sep 14 07:53:57.621989 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:57.621649    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hxw5j"
Sep 14 07:53:58.578422 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:58.578389    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-8mnfx"
Sep 14 07:53:58.597222 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:53:58.597167    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hxw5j" podStartSLOduration=2.7657876740000003 podStartE2EDuration="5.5971517s" totalImagesPullingTime="2.831364026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:53:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:53:57.642599601 +0000 UTC m=+340.099049594" watchObservedRunningTime="2026-09-14 07:53:58.5971517 +0000 UTC m=+341.053601692"
Sep 14 07:54:04.607317 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:54:04.607279    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-784955c84b-v8hmn"
Sep 14 07:54:28.631230 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:54:28.631152    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-hxw5j"
Sep 14 07:54:44.224582 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:54:44.224551    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:44.283884 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:54:44.283850    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:44.355332 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:54:44.355291    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:44.426296 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:54:44.426267    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:44.506994 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:54:44.506901    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:44.605265 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:54:44.605190    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:45.011921 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:54:45.011883    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:45.112547 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:54:45.112496    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:55.073776 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:54:55.073732    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:55.179897 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:54:55.179864    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:55.241979 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:54:55.241944    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:54:57.754977 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:54:57.754940    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:00.160971 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:00.160927    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:00.604266 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:00.604225    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:02.489173 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:02.489139    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:02.518812 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:02.518776    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-g7ddw"]
Sep 14 07:55:02.521011 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:02.520993    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-g7ddw"
Sep 14 07:55:02.525980 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:55:02.525943    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"authorino-operator-76b478bd78-g7ddw\" is forbidden: User \"system:node:ip-10-0-138-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-28.ec2.internal' and this object" podUID="65e6ce0b-ec50-46ef-bd96-f3dc3fb1bdde" pod="kuadrant-system/authorino-operator-76b478bd78-g7ddw"
Sep 14 07:55:02.527001 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:55:02.526965    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"authorino-operator-dockercfg-tktfc\" is forbidden: User \"system:node:ip-10-0-138-28.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-28.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-tktfc\"" type="*v1.Secret"
Sep 14 07:55:02.527487 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:02.527463    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 14 07:55:02.529871 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:55:02.529709    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-138-28.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-28.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Sep 14 07:55:02.537937 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:02.537911    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-g7ddw"]
Sep 14 07:55:02.627435 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:02.627394    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:02.634530 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:02.634487    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmmml\" (UniqueName: \"kubernetes.io/projected/65e6ce0b-ec50-46ef-bd96-f3dc3fb1bdde-kube-api-access-zmmml\") pod \"authorino-operator-76b478bd78-g7ddw\" (UID: \"65e6ce0b-ec50-46ef-bd96-f3dc3fb1bdde\") " pod="kuadrant-system/authorino-operator-76b478bd78-g7ddw"
Sep 14 07:55:02.735750 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:02.735710    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmmml\" (UniqueName: \"kubernetes.io/projected/65e6ce0b-ec50-46ef-bd96-f3dc3fb1bdde-kube-api-access-zmmml\") pod \"authorino-operator-76b478bd78-g7ddw\" (UID: \"65e6ce0b-ec50-46ef-bd96-f3dc3fb1bdde\") " pod="kuadrant-system/authorino-operator-76b478bd78-g7ddw"
Sep 14 07:55:02.974984 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:02.974947    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:03.475512 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:03.475460    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:03.744628 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:55:03.744551    2562 projected.go:291] Couldn't get configMap kuadrant-system/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition
Sep 14 07:55:03.744628 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:55:03.744591    2562 projected.go:196] Error preparing data for projected volume kube-api-access-zmmml for pod kuadrant-system/authorino-operator-76b478bd78-g7ddw: failed to sync configmap cache: timed out waiting for the condition
Sep 14 07:55:03.745017 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:55:03.744667    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65e6ce0b-ec50-46ef-bd96-f3dc3fb1bdde-kube-api-access-zmmml podName:65e6ce0b-ec50-46ef-bd96-f3dc3fb1bdde nodeName:}" failed. No retries permitted until 2026-09-14 07:55:04.244648498 +0000 UTC m=+406.701098468 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zmmml" (UniqueName: "kubernetes.io/projected/65e6ce0b-ec50-46ef-bd96-f3dc3fb1bdde-kube-api-access-zmmml") pod "authorino-operator-76b478bd78-g7ddw" (UID: "65e6ce0b-ec50-46ef-bd96-f3dc3fb1bdde") : failed to sync configmap cache: timed out waiting for the condition
Sep 14 07:55:03.797028 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:03.796989    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:04.001414 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:04.001330    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 14 07:55:04.001588 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:04.001418    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-tktfc\""
Sep 14 07:55:04.250190 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:04.250144    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmmml\" (UniqueName: \"kubernetes.io/projected/65e6ce0b-ec50-46ef-bd96-f3dc3fb1bdde-kube-api-access-zmmml\") pod \"authorino-operator-76b478bd78-g7ddw\" (UID: \"65e6ce0b-ec50-46ef-bd96-f3dc3fb1bdde\") " pod="kuadrant-system/authorino-operator-76b478bd78-g7ddw"
Sep 14 07:55:04.252755 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:04.252696    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmmml\" (UniqueName: \"kubernetes.io/projected/65e6ce0b-ec50-46ef-bd96-f3dc3fb1bdde-kube-api-access-zmmml\") pod \"authorino-operator-76b478bd78-g7ddw\" (UID: \"65e6ce0b-ec50-46ef-bd96-f3dc3fb1bdde\") " pod="kuadrant-system/authorino-operator-76b478bd78-g7ddw"
Sep 14 07:55:04.260899 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:04.260871    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:04.335030 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:04.334986    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-g7ddw"
Sep 14 07:55:04.626103 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:04.626013    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-g7ddw"]
Sep 14 07:55:04.630205 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:55:04.630179    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65e6ce0b_ec50_46ef_bd96_f3dc3fb1bdde.slice/crio-9dbb967c37b2851c8a1894552886bfce8f197eb2c9aa16a20932b2d62842399e WatchSource:0}: Error finding container 9dbb967c37b2851c8a1894552886bfce8f197eb2c9aa16a20932b2d62842399e: Status 404 returned error can't find the container with id 9dbb967c37b2851c8a1894552886bfce8f197eb2c9aa16a20932b2d62842399e
Sep 14 07:55:04.886802 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:04.886714    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-g7ddw" event={"ID":"65e6ce0b-ec50-46ef-bd96-f3dc3fb1bdde","Type":"ContainerStarted","Data":"9dbb967c37b2851c8a1894552886bfce8f197eb2c9aa16a20932b2d62842399e"}
Sep 14 07:55:05.433680 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:05.433642    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:05.533109 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:05.533032    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:07.361071 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:07.361033    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:07.467357 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:07.467327    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:07.902236 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:07.902136    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-g7ddw" event={"ID":"65e6ce0b-ec50-46ef-bd96-f3dc3fb1bdde","Type":"ContainerStarted","Data":"ea1db8dc76d7bf9b8f170fd2f58e778b48867ab3d9a198000ccb569940082a88"}
Sep 14 07:55:07.902412 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:07.902271    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-g7ddw"
Sep 14 07:55:07.930103 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:07.930053    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-g7ddw" podStartSLOduration=3.441859065 podStartE2EDuration="5.930038142s" totalImagesPullingTime="2.488179077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:55:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:07.927039149 +0000 UTC m=+410.383489143" watchObservedRunningTime="2026-09-14 07:55:07.930038142 +0000 UTC m=+410.386488136"
Sep 14 07:55:09.106256 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:09.106222    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-klrkj"]
Sep 14 07:55:09.109601 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:09.109580    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-klrkj"
Sep 14 07:55:09.112789 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:09.112766    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-skd5l\""
Sep 14 07:55:09.113649 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:09.113632    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 14 07:55:09.122665 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:09.122639    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-klrkj"]
Sep 14 07:55:09.198330 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:09.198286    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c2xk\" (UniqueName: \"kubernetes.io/projected/a5d25397-2d62-4dc7-93be-60428216e944-kube-api-access-8c2xk\") pod \"dns-operator-controller-manager-7b84cd7c9c-klrkj\" (UID: \"a5d25397-2d62-4dc7-93be-60428216e944\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-klrkj"
Sep 14 07:55:09.299733 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:09.299695    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c2xk\" (UniqueName: \"kubernetes.io/projected/a5d25397-2d62-4dc7-93be-60428216e944-kube-api-access-8c2xk\") pod \"dns-operator-controller-manager-7b84cd7c9c-klrkj\" (UID: \"a5d25397-2d62-4dc7-93be-60428216e944\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-klrkj"
Sep 14 07:55:09.328143 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:09.328111    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c2xk\" (UniqueName: \"kubernetes.io/projected/a5d25397-2d62-4dc7-93be-60428216e944-kube-api-access-8c2xk\") pod \"dns-operator-controller-manager-7b84cd7c9c-klrkj\" (UID: \"a5d25397-2d62-4dc7-93be-60428216e944\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-klrkj"
Sep 14 07:55:09.419908 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:09.419811    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-klrkj"
Sep 14 07:55:09.711075 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:09.711045    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-klrkj"]
Sep 14 07:55:09.712449 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:55:09.712422    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5d25397_2d62_4dc7_93be_60428216e944.slice/crio-db9c1151c30851baca68c90e39146b024e5db317f52c50a869e45addbf9ec363 WatchSource:0}: Error finding container db9c1151c30851baca68c90e39146b024e5db317f52c50a869e45addbf9ec363: Status 404 returned error can't find the container with id db9c1151c30851baca68c90e39146b024e5db317f52c50a869e45addbf9ec363
Sep 14 07:55:09.912030 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:09.911998    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-klrkj" event={"ID":"a5d25397-2d62-4dc7-93be-60428216e944","Type":"ContainerStarted","Data":"db9c1151c30851baca68c90e39146b024e5db317f52c50a869e45addbf9ec363"}
Sep 14 07:55:10.369061 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:10.369009    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:10.488980 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:10.488948    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:11.921886 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:11.921850    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-klrkj" event={"ID":"a5d25397-2d62-4dc7-93be-60428216e944","Type":"ContainerStarted","Data":"19aef72c3ffa8bf86a171b4415a1549af07862f9d0bda11ef72a8a67fa14de7e"}
Sep 14 07:55:11.922256 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:11.921919    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-klrkj"
Sep 14 07:55:11.941946 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:11.941896    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-klrkj" podStartSLOduration=1.297132766 podStartE2EDuration="2.941878564s" totalImagesPullingTime="1.644745798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:55:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:11.94095802 +0000 UTC m=+414.397408042" watchObservedRunningTime="2026-09-14 07:55:11.941878564 +0000 UTC m=+414.398328557"
Sep 14 07:55:12.027475 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:12.027447    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:13.838433 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:13.838401    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:14.159072 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:14.158995    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:14.260252 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:14.260214    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:14.361090 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:14.361053    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:14.423304 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:14.423219    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:14.549728 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:14.549695    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:14.651291 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:14.651260    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:17.558185 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:17.558145    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:17.628591 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:17.628556    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:18.909399 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:18.909369    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-g7ddw"
Sep 14 07:55:19.299935 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:19.299902    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:19.405329 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:19.405295    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:19.517378 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:19.517340    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:22.928199 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:22.928166    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-klrkj"
Sep 14 07:55:24.463318 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:24.463281    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-m8ntq"]
Sep 14 07:55:24.466731 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:24.466714    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-m8ntq"
Sep 14 07:55:24.468715 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:24.468692    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-xpfct\""
Sep 14 07:55:24.491283 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:24.491247    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-m8ntq"]
Sep 14 07:55:24.577171 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:24.577137    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-m8ntq"]
Sep 14 07:55:24.577420 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:55:24.577401    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-fwnhq], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-m8ntq" podUID="f16b901e-49bd-4f64-9dac-a445e4ad8b54"
Sep 14 07:55:24.633879 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:24.633840    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwnhq\" (UniqueName: \"kubernetes.io/projected/f16b901e-49bd-4f64-9dac-a445e4ad8b54-kube-api-access-fwnhq\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-m8ntq\" (UID: \"f16b901e-49bd-4f64-9dac-a445e4ad8b54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-m8ntq"
Sep 14 07:55:24.634041 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:24.633887    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f16b901e-49bd-4f64-9dac-a445e4ad8b54-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-m8ntq\" (UID: \"f16b901e-49bd-4f64-9dac-a445e4ad8b54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-m8ntq"
Sep 14 07:55:24.735337 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:24.735237    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwnhq\" (UniqueName: \"kubernetes.io/projected/f16b901e-49bd-4f64-9dac-a445e4ad8b54-kube-api-access-fwnhq\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-m8ntq\" (UID: \"f16b901e-49bd-4f64-9dac-a445e4ad8b54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-m8ntq"
Sep 14 07:55:24.735337 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:24.735302    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f16b901e-49bd-4f64-9dac-a445e4ad8b54-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-m8ntq\" (UID: \"f16b901e-49bd-4f64-9dac-a445e4ad8b54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-m8ntq"
Sep 14 07:55:24.735747 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:24.735725    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f16b901e-49bd-4f64-9dac-a445e4ad8b54-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-m8ntq\" (UID: \"f16b901e-49bd-4f64-9dac-a445e4ad8b54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-m8ntq"
Sep 14 07:55:24.744025 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:24.743999    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwnhq\" (UniqueName: \"kubernetes.io/projected/f16b901e-49bd-4f64-9dac-a445e4ad8b54-kube-api-access-fwnhq\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-m8ntq\" (UID: \"f16b901e-49bd-4f64-9dac-a445e4ad8b54\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-m8ntq"
Sep 14 07:55:24.976252 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:24.976211    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-m8ntq"
Sep 14 07:55:24.983595 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:24.983568    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-m8ntq"
Sep 14 07:55:25.139232 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:25.139140    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f16b901e-49bd-4f64-9dac-a445e4ad8b54-extensions-socket-volume\") pod \"f16b901e-49bd-4f64-9dac-a445e4ad8b54\" (UID: \"f16b901e-49bd-4f64-9dac-a445e4ad8b54\") "
Sep 14 07:55:25.139232 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:25.139199    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwnhq\" (UniqueName: \"kubernetes.io/projected/f16b901e-49bd-4f64-9dac-a445e4ad8b54-kube-api-access-fwnhq\") pod \"f16b901e-49bd-4f64-9dac-a445e4ad8b54\" (UID: \"f16b901e-49bd-4f64-9dac-a445e4ad8b54\") "
Sep 14 07:55:25.139471 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:25.139374    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f16b901e-49bd-4f64-9dac-a445e4ad8b54-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f16b901e-49bd-4f64-9dac-a445e4ad8b54" (UID: "f16b901e-49bd-4f64-9dac-a445e4ad8b54"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:55:25.141320 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:25.141287    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f16b901e-49bd-4f64-9dac-a445e4ad8b54-kube-api-access-fwnhq" (OuterVolumeSpecName: "kube-api-access-fwnhq") pod "f16b901e-49bd-4f64-9dac-a445e4ad8b54" (UID: "f16b901e-49bd-4f64-9dac-a445e4ad8b54"). InnerVolumeSpecName "kube-api-access-fwnhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:55:25.240202 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:25.240163    2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f16b901e-49bd-4f64-9dac-a445e4ad8b54-extensions-socket-volume\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:55:25.240202 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:25.240193    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwnhq\" (UniqueName: \"kubernetes.io/projected/f16b901e-49bd-4f64-9dac-a445e4ad8b54-kube-api-access-fwnhq\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:55:25.461702 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:25.461644    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-m8ntq"]
Sep 14 07:55:25.474154 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:25.474120    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-m8ntq"]
Sep 14 07:55:25.528558 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:25.528521    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5"]
Sep 14 07:55:25.531773 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:25.531756    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5"
Sep 14 07:55:25.548733 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:25.548703    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5"]
Sep 14 07:55:25.645293 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:25.645254    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8701b87-012c-44ac-9bd0-d1b147fc2be9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s9ck5\" (UID: \"c8701b87-012c-44ac-9bd0-d1b147fc2be9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5"
Sep 14 07:55:25.645458 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:25.645304    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp55p\" (UniqueName: \"kubernetes.io/projected/c8701b87-012c-44ac-9bd0-d1b147fc2be9-kube-api-access-jp55p\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s9ck5\" (UID: \"c8701b87-012c-44ac-9bd0-d1b147fc2be9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5"
Sep 14 07:55:25.746573 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:25.746462    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8701b87-012c-44ac-9bd0-d1b147fc2be9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s9ck5\" (UID: \"c8701b87-012c-44ac-9bd0-d1b147fc2be9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5"
Sep 14 07:55:25.746573 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:25.746538    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp55p\" (UniqueName: \"kubernetes.io/projected/c8701b87-012c-44ac-9bd0-d1b147fc2be9-kube-api-access-jp55p\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s9ck5\" (UID: \"c8701b87-012c-44ac-9bd0-d1b147fc2be9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5"
Sep 14 07:55:25.746906 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:25.746881    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8701b87-012c-44ac-9bd0-d1b147fc2be9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s9ck5\" (UID: \"c8701b87-012c-44ac-9bd0-d1b147fc2be9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5"
Sep 14 07:55:25.757753 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:25.757719    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp55p\" (UniqueName: \"kubernetes.io/projected/c8701b87-012c-44ac-9bd0-d1b147fc2be9-kube-api-access-jp55p\") pod \"kuadrant-operator-controller-manager-76f8f686bf-s9ck5\" (UID: \"c8701b87-012c-44ac-9bd0-d1b147fc2be9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5"
Sep 14 07:55:25.799247 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:25.799207    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:25.851531 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:25.851472    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5"
Sep 14 07:55:25.901663 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:25.901631    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:25.980688 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:25.980653    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-m8ntq"
Sep 14 07:55:25.983336 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:55:25.983312    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-m8ntq\" is forbidden: User \"system:node:ip-10-0-138-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-28.ec2.internal' and this object" podUID="f16b901e-49bd-4f64-9dac-a445e4ad8b54" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-m8ntq"
Sep 14 07:55:25.989274 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:55:25.989245    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-m8ntq\" is forbidden: User \"system:node:ip-10-0-138-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-28.ec2.internal' and this object" podUID="f16b901e-49bd-4f64-9dac-a445e4ad8b54" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-m8ntq"
Sep 14 07:55:26.112334 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:26.112261    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f16b901e-49bd-4f64-9dac-a445e4ad8b54" path="/var/lib/kubelet/pods/f16b901e-49bd-4f64-9dac-a445e4ad8b54/volumes"
Sep 14 07:55:26.137773 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:26.137739    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5"]
Sep 14 07:55:26.141192 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:55:26.141168    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8701b87_012c_44ac_9bd0_d1b147fc2be9.slice/crio-91634ce3f4c4e9fd1dc811fffaff3eda9be64ce5ebe8b8db0740306f5c73f522 WatchSource:0}: Error finding container 91634ce3f4c4e9fd1dc811fffaff3eda9be64ce5ebe8b8db0740306f5c73f522: Status 404 returned error can't find the container with id 91634ce3f4c4e9fd1dc811fffaff3eda9be64ce5ebe8b8db0740306f5c73f522
Sep 14 07:55:26.987279 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:26.987243    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5" event={"ID":"c8701b87-012c-44ac-9bd0-d1b147fc2be9","Type":"ContainerStarted","Data":"91634ce3f4c4e9fd1dc811fffaff3eda9be64ce5ebe8b8db0740306f5c73f522"}
Sep 14 07:55:30.003279 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:30.003237    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5" event={"ID":"c8701b87-012c-44ac-9bd0-d1b147fc2be9","Type":"ContainerStarted","Data":"e260a12c87fac9a706fa099c00c55ef683c7ed9e958ab10fe4cca8864d98e76f"}
Sep 14 07:55:30.003738 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:30.003305    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5"
Sep 14 07:55:30.026070 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:30.026023    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5" podStartSLOduration=1.662520877 podStartE2EDuration="5.026008605s" totalImagesPullingTime="3.363487728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:30.025402565 +0000 UTC m=+432.481852557" watchObservedRunningTime="2026-09-14 07:55:30.026008605 +0000 UTC m=+432.482458597"
Sep 14 07:55:30.141257 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:30.141216    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:30.241999 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:30.241845    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:33.941597 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:33.941561    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:34.041938 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:34.041905    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:34.141597 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:34.141545    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:34.242889 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:34.242792    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:38.179512 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:38.179467    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5"]
Sep 14 07:55:38.179977 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:38.179744    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5" podUID="c8701b87-012c-44ac-9bd0-d1b147fc2be9" containerName="manager" containerID="cri-o://e260a12c87fac9a706fa099c00c55ef683c7ed9e958ab10fe4cca8864d98e76f" gracePeriod=10
Sep 14 07:55:38.181706 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:38.181678    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5"
Sep 14 07:55:38.423968 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:38.423939    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5"
Sep 14 07:55:38.450791 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:38.450757    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jp55p\" (UniqueName: \"kubernetes.io/projected/c8701b87-012c-44ac-9bd0-d1b147fc2be9-kube-api-access-jp55p\") pod \"c8701b87-012c-44ac-9bd0-d1b147fc2be9\" (UID: \"c8701b87-012c-44ac-9bd0-d1b147fc2be9\") "
Sep 14 07:55:38.450963 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:38.450834    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8701b87-012c-44ac-9bd0-d1b147fc2be9-extensions-socket-volume\") pod \"c8701b87-012c-44ac-9bd0-d1b147fc2be9\" (UID: \"c8701b87-012c-44ac-9bd0-d1b147fc2be9\") "
Sep 14 07:55:38.451427 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:38.451392    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8701b87-012c-44ac-9bd0-d1b147fc2be9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c8701b87-012c-44ac-9bd0-d1b147fc2be9" (UID: "c8701b87-012c-44ac-9bd0-d1b147fc2be9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:55:38.453465 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:38.453412    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8701b87-012c-44ac-9bd0-d1b147fc2be9-kube-api-access-jp55p" (OuterVolumeSpecName: "kube-api-access-jp55p") pod "c8701b87-012c-44ac-9bd0-d1b147fc2be9" (UID: "c8701b87-012c-44ac-9bd0-d1b147fc2be9"). InnerVolumeSpecName "kube-api-access-jp55p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:55:38.551964 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:38.551920    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jp55p\" (UniqueName: \"kubernetes.io/projected/c8701b87-012c-44ac-9bd0-d1b147fc2be9-kube-api-access-jp55p\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:55:38.551964 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:38.551957    2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8701b87-012c-44ac-9bd0-d1b147fc2be9-extensions-socket-volume\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 07:55:39.044401 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:39.044359    2562 generic.go:356] "Generic (PLEG): container finished" podID="c8701b87-012c-44ac-9bd0-d1b147fc2be9" containerID="e260a12c87fac9a706fa099c00c55ef683c7ed9e958ab10fe4cca8864d98e76f" exitCode=0
Sep 14 07:55:39.044624 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:39.044408    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5"
Sep 14 07:55:39.044624 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:39.044455    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5" event={"ID":"c8701b87-012c-44ac-9bd0-d1b147fc2be9","Type":"ContainerDied","Data":"e260a12c87fac9a706fa099c00c55ef683c7ed9e958ab10fe4cca8864d98e76f"}
Sep 14 07:55:39.044624 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:39.044538    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5" event={"ID":"c8701b87-012c-44ac-9bd0-d1b147fc2be9","Type":"ContainerDied","Data":"91634ce3f4c4e9fd1dc811fffaff3eda9be64ce5ebe8b8db0740306f5c73f522"}
Sep 14 07:55:39.044624 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:39.044574    2562 scope.go:117] "RemoveContainer" containerID="e260a12c87fac9a706fa099c00c55ef683c7ed9e958ab10fe4cca8864d98e76f"
Sep 14 07:55:39.053607 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:39.053587    2562 scope.go:117] "RemoveContainer" containerID="e260a12c87fac9a706fa099c00c55ef683c7ed9e958ab10fe4cca8864d98e76f"
Sep 14 07:55:39.053876 ip-10-0-138-28 kubenswrapper[2562]: E0914 07:55:39.053858    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e260a12c87fac9a706fa099c00c55ef683c7ed9e958ab10fe4cca8864d98e76f\": container with ID starting with e260a12c87fac9a706fa099c00c55ef683c7ed9e958ab10fe4cca8864d98e76f not found: ID does not exist" containerID="e260a12c87fac9a706fa099c00c55ef683c7ed9e958ab10fe4cca8864d98e76f"
Sep 14 07:55:39.053935 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:39.053883    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e260a12c87fac9a706fa099c00c55ef683c7ed9e958ab10fe4cca8864d98e76f"} err="failed to get container status \"e260a12c87fac9a706fa099c00c55ef683c7ed9e958ab10fe4cca8864d98e76f\": rpc error: code = NotFound desc = could not find container \"e260a12c87fac9a706fa099c00c55ef683c7ed9e958ab10fe4cca8864d98e76f\": container with ID starting with e260a12c87fac9a706fa099c00c55ef683c7ed9e958ab10fe4cca8864d98e76f not found: ID does not exist"
Sep 14 07:55:39.068604 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:39.068580    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5"]
Sep 14 07:55:39.075013 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:39.074986    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-s9ck5"]
Sep 14 07:55:39.549621 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:39.549587    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:39.655120 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:39.655088    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:39.749894 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:39.749859    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:39.850102 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:39.850006    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:39.950741 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:39.950712    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:40.051025 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:40.050991    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:40.112464 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:40.112382    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8701b87-012c-44ac-9bd0-d1b147fc2be9" path="/var/lib/kubelet/pods/c8701b87-012c-44ac-9bd0-d1b147fc2be9/volumes"
Sep 14 07:55:45.847158 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:45.847097    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:45.909049 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:45.909005    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:54.570696 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.570662    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"]
Sep 14 07:55:54.571136 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.571013    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8701b87-012c-44ac-9bd0-d1b147fc2be9" containerName="manager"
Sep 14 07:55:54.571136 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.571027    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8701b87-012c-44ac-9bd0-d1b147fc2be9" containerName="manager"
Sep 14 07:55:54.571136 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.571085    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8701b87-012c-44ac-9bd0-d1b147fc2be9" containerName="manager"
Sep 14 07:55:54.573890 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.573872    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.577491 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.577467    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-2mhds\""
Sep 14 07:55:54.586229 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.586200    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"]
Sep 14 07:55:54.600936 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.600903    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:54.689754 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.689713    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f6db21f-387c-48d5-a12f-9f809fbd87d2-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.689754 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.689758    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f6db21f-387c-48d5-a12f-9f809fbd87d2-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.689965 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.689849    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f6db21f-387c-48d5-a12f-9f809fbd87d2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.689965 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.689900    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f6db21f-387c-48d5-a12f-9f809fbd87d2-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.689965 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.689927    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f6db21f-387c-48d5-a12f-9f809fbd87d2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.690065 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.689980    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f6db21f-387c-48d5-a12f-9f809fbd87d2-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.690065 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.689995    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9cx5\" (UniqueName: \"kubernetes.io/projected/5f6db21f-387c-48d5-a12f-9f809fbd87d2-kube-api-access-k9cx5\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.690065 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.690024    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f6db21f-387c-48d5-a12f-9f809fbd87d2-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.690065 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.690042    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f6db21f-387c-48d5-a12f-9f809fbd87d2-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.697265 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.697234    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:55:54.790540 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.790488    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f6db21f-387c-48d5-a12f-9f809fbd87d2-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.790540 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.790540    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9cx5\" (UniqueName: \"kubernetes.io/projected/5f6db21f-387c-48d5-a12f-9f809fbd87d2-kube-api-access-k9cx5\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.790720 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.790648    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f6db21f-387c-48d5-a12f-9f809fbd87d2-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.790720 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.790682    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f6db21f-387c-48d5-a12f-9f809fbd87d2-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.790791 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.790729    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f6db21f-387c-48d5-a12f-9f809fbd87d2-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.790791 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.790749    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f6db21f-387c-48d5-a12f-9f809fbd87d2-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.790791 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.790787    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f6db21f-387c-48d5-a12f-9f809fbd87d2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.790931 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.790826    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f6db21f-387c-48d5-a12f-9f809fbd87d2-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.790931 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.790857    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f6db21f-387c-48d5-a12f-9f809fbd87d2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.790931 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.790875    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f6db21f-387c-48d5-a12f-9f809fbd87d2-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.791139 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.791116    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f6db21f-387c-48d5-a12f-9f809fbd87d2-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.791211 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.791182    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f6db21f-387c-48d5-a12f-9f809fbd87d2-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.791271 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.791247    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f6db21f-387c-48d5-a12f-9f809fbd87d2-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.791788 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.791767    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f6db21f-387c-48d5-a12f-9f809fbd87d2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.792828 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.792811    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f6db21f-387c-48d5-a12f-9f809fbd87d2-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.793161 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.793142    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f6db21f-387c-48d5-a12f-9f809fbd87d2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.799079 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.799055    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9cx5\" (UniqueName: \"kubernetes.io/projected/5f6db21f-387c-48d5-a12f-9f809fbd87d2-kube-api-access-k9cx5\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.799703 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.799680    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f6db21f-387c-48d5-a12f-9f809fbd87d2-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zdnqg\" (UID: \"5f6db21f-387c-48d5-a12f-9f809fbd87d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:54.886610 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:54.886491    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:55.161575 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:55.161551    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"]
Sep 14 07:55:55.164020 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:55:55.163986    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f6db21f_387c_48d5_a12f_9f809fbd87d2.slice/crio-858f5c15b22b99be2fc692d50c3c8687936e1952267d508c8c77475ea12d8288 WatchSource:0}: Error finding container 858f5c15b22b99be2fc692d50c3c8687936e1952267d508c8c77475ea12d8288: Status 404 returned error can't find the container with id 858f5c15b22b99be2fc692d50c3c8687936e1952267d508c8c77475ea12d8288
Sep 14 07:55:55.166189 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:55.166159    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 07:55:55.166253 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:55.166224    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 07:55:55.166290 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:55.166250    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 07:55:56.120460 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:56.120375    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg" event={"ID":"5f6db21f-387c-48d5-a12f-9f809fbd87d2","Type":"ContainerStarted","Data":"d5bc604f5bb490b86410539b01f62733aab1f25bb859415d5dbf1ce2f85411b1"}
Sep 14 07:55:56.120460 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:56.120433    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg" event={"ID":"5f6db21f-387c-48d5-a12f-9f809fbd87d2","Type":"ContainerStarted","Data":"858f5c15b22b99be2fc692d50c3c8687936e1952267d508c8c77475ea12d8288"}
Sep 14 07:55:56.144046 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:56.143989    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg" podStartSLOduration=2.143972347 podStartE2EDuration="2.143972347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:55:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:56.142429187 +0000 UTC m=+458.598879180" watchObservedRunningTime="2026-09-14 07:55:56.143972347 +0000 UTC m=+458.600422339"
Sep 14 07:55:56.887462 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:56.887411    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:56.892094 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:56.892067    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:57.125449 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:57.125415    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:55:57.126495 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:55:57.126475    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:56:07.500117 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:07.500077    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:56:07.609424 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:07.609380    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:56:07.753742 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:07.753651    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:56:07.838884 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:07.838852    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 07:56:07.841135 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:07.841118    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"
Sep 14 07:56:07.843371 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:07.843349    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 14 07:56:07.843492 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:07.843438    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-v4tmp\""
Sep 14 07:56:07.854323 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:07.854300    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 07:56:07.885082 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:07.885049    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 07:56:07.904976 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:07.904944    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l7nb\" (UniqueName: \"kubernetes.io/projected/51555902-c387-45cb-ac73-3f43dc26962a-kube-api-access-5l7nb\") pod \"limitador-limitador-84994f7ffc-rv6tc\" (UID: \"51555902-c387-45cb-ac73-3f43dc26962a\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"
Sep 14 07:56:07.904976 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:07.904976    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/51555902-c387-45cb-ac73-3f43dc26962a-config-file\") pod \"limitador-limitador-84994f7ffc-rv6tc\" (UID: \"51555902-c387-45cb-ac73-3f43dc26962a\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"
Sep 14 07:56:08.005965 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:08.005865    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l7nb\" (UniqueName: \"kubernetes.io/projected/51555902-c387-45cb-ac73-3f43dc26962a-kube-api-access-5l7nb\") pod \"limitador-limitador-84994f7ffc-rv6tc\" (UID: \"51555902-c387-45cb-ac73-3f43dc26962a\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"
Sep 14 07:56:08.005965 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:08.005911    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/51555902-c387-45cb-ac73-3f43dc26962a-config-file\") pod \"limitador-limitador-84994f7ffc-rv6tc\" (UID: \"51555902-c387-45cb-ac73-3f43dc26962a\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"
Sep 14 07:56:08.006595 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:08.006571    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/51555902-c387-45cb-ac73-3f43dc26962a-config-file\") pod \"limitador-limitador-84994f7ffc-rv6tc\" (UID: \"51555902-c387-45cb-ac73-3f43dc26962a\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"
Sep 14 07:56:08.016870 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:08.016848    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l7nb\" (UniqueName: \"kubernetes.io/projected/51555902-c387-45cb-ac73-3f43dc26962a-kube-api-access-5l7nb\") pod \"limitador-limitador-84994f7ffc-rv6tc\" (UID: \"51555902-c387-45cb-ac73-3f43dc26962a\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"
Sep 14 07:56:08.151091 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:08.151053    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"
Sep 14 07:56:08.346578 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:08.346469    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:56:08.420409 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:08.420247    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 07:56:08.423742 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:08.423712    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:56:08.424173 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:56:08.424134    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51555902_c387_45cb_ac73_3f43dc26962a.slice/crio-29ae70e29311d2d8c6613d59a9dd244d102503c588d51d0d25da6e8301dda3db WatchSource:0}: Error finding container 29ae70e29311d2d8c6613d59a9dd244d102503c588d51d0d25da6e8301dda3db: Status 404 returned error can't find the container with id 29ae70e29311d2d8c6613d59a9dd244d102503c588d51d0d25da6e8301dda3db
Sep 14 07:56:09.181486 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:09.181446    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-rv6tc" event={"ID":"51555902-c387-45cb-ac73-3f43dc26962a","Type":"ContainerStarted","Data":"29ae70e29311d2d8c6613d59a9dd244d102503c588d51d0d25da6e8301dda3db"}
Sep 14 07:56:10.187459 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:10.187425    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-rv6tc" event={"ID":"51555902-c387-45cb-ac73-3f43dc26962a","Type":"ContainerStarted","Data":"d7a8294ab68d3e8055a66bc002a2be3d1059053b9375cac07fdcf6aad4da7506"}
Sep 14 07:56:10.187873 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:10.187551    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"
Sep 14 07:56:10.205390 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:10.205337    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-rv6tc" podStartSLOduration=2.350916749 podStartE2EDuration="3.205320205s" totalImagesPullingTime="854.403456ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:56:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:56:10.203602803 +0000 UTC m=+472.660052796" watchObservedRunningTime="2026-09-14 07:56:10.205320205 +0000 UTC m=+472.661770198"
Sep 14 07:56:21.192159 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:21.192129    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"
Sep 14 07:56:23.487159 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:23.487118    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:56:23.582801 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:23.582766    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:56:24.081514 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:24.081482    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:56:35.808732 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:35.808698    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:56:36.107101 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:36.106986    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:56:43.850947 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:43.850907    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:56:43.904689 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:43.904654    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:56:44.087615 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:44.087573    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:56:44.239243 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:44.239206    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:56:47.841047 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:47.841005    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:56:47.941143 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:47.941108    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:56:59.128300 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:56:59.128262    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:04.460726 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:04.460688    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:04.515943 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:04.515913    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:19.349134 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:19.349041    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:19.451450 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:19.451388    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:20.822051 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:20.822011    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:20.932945 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:20.932901    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:21.314210 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:21.314170    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:21.415797 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:21.415762    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:21.592169 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:21.592077    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:21.695583 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:21.695551    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:21.837558 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:21.837524    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 07:57:31.550596 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:31.550557    2562 generic.go:356] "Generic (PLEG): container finished" podID="5f6db21f-387c-48d5-a12f-9f809fbd87d2" containerID="d5bc604f5bb490b86410539b01f62733aab1f25bb859415d5dbf1ce2f85411b1" exitCode=0
Sep 14 07:57:31.551002 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:31.550630    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg" event={"ID":"5f6db21f-387c-48d5-a12f-9f809fbd87d2","Type":"ContainerDied","Data":"d5bc604f5bb490b86410539b01f62733aab1f25bb859415d5dbf1ce2f85411b1"}
Sep 14 07:57:31.551113 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:31.551093    2562 scope.go:117] "RemoveContainer" containerID="d5bc604f5bb490b86410539b01f62733aab1f25bb859415d5dbf1ce2f85411b1"
Sep 14 07:57:31.552108 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:31.552080    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 07:57:31.552170 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:31.552149    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 07:57:31.552204 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:31.552179    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 07:57:31.625660 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:31.625627    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:32.314738 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:32.314708    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:32.409935 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:32.409896    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:32.560421 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:32.560377    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg" event={"ID":"5f6db21f-387c-48d5-a12f-9f809fbd87d2","Type":"ContainerStarted","Data":"9a513ea2a89a89ec347b2f40962a9b1f93f447ccbdabdfe6b4ff5a8e96cff4d5"}
Sep 14 07:57:32.886976 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:32.886938    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:57:32.887168 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:32.887141    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-zdnqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Sep 14 07:57:32.887227 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:32.887203    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg" podUID="5f6db21f-387c-48d5-a12f-9f809fbd87d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Sep 14 07:57:33.887672 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:33.887633    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-zdnqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Sep 14 07:57:33.888065 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:33.887694    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg" podUID="5f6db21f-387c-48d5-a12f-9f809fbd87d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Sep 14 07:57:34.887797 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:34.887756    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-zdnqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Sep 14 07:57:34.888266 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:34.887824    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg" podUID="5f6db21f-387c-48d5-a12f-9f809fbd87d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Sep 14 07:57:35.887515 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:35.887473    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-zdnqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Sep 14 07:57:35.887704 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:35.887567    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg" podUID="5f6db21f-387c-48d5-a12f-9f809fbd87d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Sep 14 07:57:36.887329 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:36.887287    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-zdnqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Sep 14 07:57:36.887731 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:36.887344    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg" podUID="5f6db21f-387c-48d5-a12f-9f809fbd87d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Sep 14 07:57:37.890825 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:37.890792    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:57:37.891225 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:37.891201    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:57:37.891794 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:37.891770    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zdnqg"
Sep 14 07:57:42.534333 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:42.534303    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:43.195828 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:43.195792    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 07:57:53.939422 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:53.939380    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:57:54.725955 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:57:54.725919    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 07:58:15.308082 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.308024    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"]
Sep 14 07:58:15.310145 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.310127    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:15.313317 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.313296    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 14 07:58:15.313497 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.313477    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7j9jg\""
Sep 14 07:58:15.313608 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.313478    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 14 07:58:15.313608 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.313478    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 14 07:58:15.321970 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.321945    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"]
Sep 14 07:58:15.394606 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.394576    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:15.397323 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.397300    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87434236-fa77-40b1-a197-b0f4ca8f22bc-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-266jz\" (UID: \"87434236-fa77-40b1-a197-b0f4ca8f22bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:15.397446 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.397348    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87434236-fa77-40b1-a197-b0f4ca8f22bc-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-266jz\" (UID: \"87434236-fa77-40b1-a197-b0f4ca8f22bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:15.397446 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.397407    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87434236-fa77-40b1-a197-b0f4ca8f22bc-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-266jz\" (UID: \"87434236-fa77-40b1-a197-b0f4ca8f22bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:15.397593 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.397461    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87434236-fa77-40b1-a197-b0f4ca8f22bc-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-266jz\" (UID: \"87434236-fa77-40b1-a197-b0f4ca8f22bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:15.397593 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.397522    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knhx2\" (UniqueName: \"kubernetes.io/projected/87434236-fa77-40b1-a197-b0f4ca8f22bc-kube-api-access-knhx2\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-266jz\" (UID: \"87434236-fa77-40b1-a197-b0f4ca8f22bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:15.397593 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.397556    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87434236-fa77-40b1-a197-b0f4ca8f22bc-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-266jz\" (UID: \"87434236-fa77-40b1-a197-b0f4ca8f22bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:15.498301 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.498263    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87434236-fa77-40b1-a197-b0f4ca8f22bc-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-266jz\" (UID: \"87434236-fa77-40b1-a197-b0f4ca8f22bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:15.498476 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.498321    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87434236-fa77-40b1-a197-b0f4ca8f22bc-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-266jz\" (UID: \"87434236-fa77-40b1-a197-b0f4ca8f22bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:15.498476 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.498350    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87434236-fa77-40b1-a197-b0f4ca8f22bc-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-266jz\" (UID: \"87434236-fa77-40b1-a197-b0f4ca8f22bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:15.498476 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.498370    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knhx2\" (UniqueName: \"kubernetes.io/projected/87434236-fa77-40b1-a197-b0f4ca8f22bc-kube-api-access-knhx2\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-266jz\" (UID: \"87434236-fa77-40b1-a197-b0f4ca8f22bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:15.498476 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.498397    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87434236-fa77-40b1-a197-b0f4ca8f22bc-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-266jz\" (UID: \"87434236-fa77-40b1-a197-b0f4ca8f22bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:15.498476 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.498441    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87434236-fa77-40b1-a197-b0f4ca8f22bc-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-266jz\" (UID: \"87434236-fa77-40b1-a197-b0f4ca8f22bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:15.498826 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.498805    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87434236-fa77-40b1-a197-b0f4ca8f22bc-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-266jz\" (UID: \"87434236-fa77-40b1-a197-b0f4ca8f22bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:15.498902 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.498857    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87434236-fa77-40b1-a197-b0f4ca8f22bc-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-266jz\" (UID: \"87434236-fa77-40b1-a197-b0f4ca8f22bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:15.498958 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.498939    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87434236-fa77-40b1-a197-b0f4ca8f22bc-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-266jz\" (UID: \"87434236-fa77-40b1-a197-b0f4ca8f22bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:15.500706 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.500684    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87434236-fa77-40b1-a197-b0f4ca8f22bc-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-266jz\" (UID: \"87434236-fa77-40b1-a197-b0f4ca8f22bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:15.500818 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.500800    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87434236-fa77-40b1-a197-b0f4ca8f22bc-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-266jz\" (UID: \"87434236-fa77-40b1-a197-b0f4ca8f22bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:15.512494 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.512458    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knhx2\" (UniqueName: \"kubernetes.io/projected/87434236-fa77-40b1-a197-b0f4ca8f22bc-kube-api-access-knhx2\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-266jz\" (UID: \"87434236-fa77-40b1-a197-b0f4ca8f22bc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:15.620529 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.620418    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:15.893155 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:15.893126    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"]
Sep 14 07:58:15.895710 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:58:15.895677    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87434236_fa77_40b1_a197_b0f4ca8f22bc.slice/crio-207d3a4940fb29852edacec65b8eb6154472a25941e2df5beda427b3c179993b WatchSource:0}: Error finding container 207d3a4940fb29852edacec65b8eb6154472a25941e2df5beda427b3c179993b: Status 404 returned error can't find the container with id 207d3a4940fb29852edacec65b8eb6154472a25941e2df5beda427b3c179993b
Sep 14 07:58:16.394483 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:16.394434    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 07:58:16.757281 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:16.757237    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz" event={"ID":"87434236-fa77-40b1-a197-b0f4ca8f22bc","Type":"ContainerStarted","Data":"207d3a4940fb29852edacec65b8eb6154472a25941e2df5beda427b3c179993b"}
Sep 14 07:58:17.763327 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:17.763292    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz" event={"ID":"87434236-fa77-40b1-a197-b0f4ca8f22bc","Type":"ContainerStarted","Data":"c4af4e9e830ce9c00878af9b39a079251647ac84ab1fa2a0804529b800627516"}
Sep 14 07:58:17.781697 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:17.781647    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz" podStartSLOduration=1.866072356 podStartE2EDuration="2.781632099s" totalImagesPullingTime="915.559743ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:58:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:58:17.779731157 +0000 UTC m=+600.236181162" watchObservedRunningTime="2026-09-14 07:58:17.781632099 +0000 UTC m=+600.238082091"
Sep 14 07:58:19.499456 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:19.499409    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:21.104496 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:21.104457    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 07:58:22.811933 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:22.811892    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:22.864260 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:22.864229    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:25.621069 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:25.621031    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:25.621496 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:25.621094    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:25.633746 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:25.633716    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:25.808158 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:25.808129    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-266jz"
Sep 14 07:58:32.894094 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:32.894060    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"]
Sep 14 07:58:32.899392 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:32.899358    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:32.901664 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:32.901641    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 14 07:58:32.907835 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:32.907809    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"]
Sep 14 07:58:32.942940 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:32.942901    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5400c3d5-1766-439c-947d-5da15cf447ee-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47\" (UID: \"5400c3d5-1766-439c-947d-5da15cf447ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:32.942940 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:32.942942    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5400c3d5-1766-439c-947d-5da15cf447ee-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47\" (UID: \"5400c3d5-1766-439c-947d-5da15cf447ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:32.943276 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:32.942979    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5400c3d5-1766-439c-947d-5da15cf447ee-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47\" (UID: \"5400c3d5-1766-439c-947d-5da15cf447ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:32.943276 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:32.943020    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5400c3d5-1766-439c-947d-5da15cf447ee-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47\" (UID: \"5400c3d5-1766-439c-947d-5da15cf447ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:32.943276 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:32.943093    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5400c3d5-1766-439c-947d-5da15cf447ee-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47\" (UID: \"5400c3d5-1766-439c-947d-5da15cf447ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:32.943276 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:32.943136    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-298nr\" (UniqueName: \"kubernetes.io/projected/5400c3d5-1766-439c-947d-5da15cf447ee-kube-api-access-298nr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47\" (UID: \"5400c3d5-1766-439c-947d-5da15cf447ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:33.043953 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:33.043915    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5400c3d5-1766-439c-947d-5da15cf447ee-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47\" (UID: \"5400c3d5-1766-439c-947d-5da15cf447ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:33.044147 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:33.043965    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5400c3d5-1766-439c-947d-5da15cf447ee-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47\" (UID: \"5400c3d5-1766-439c-947d-5da15cf447ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:33.044147 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:33.043998    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-298nr\" (UniqueName: \"kubernetes.io/projected/5400c3d5-1766-439c-947d-5da15cf447ee-kube-api-access-298nr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47\" (UID: \"5400c3d5-1766-439c-947d-5da15cf447ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:33.044147 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:33.044023    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5400c3d5-1766-439c-947d-5da15cf447ee-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47\" (UID: \"5400c3d5-1766-439c-947d-5da15cf447ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:33.044147 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:33.044044    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5400c3d5-1766-439c-947d-5da15cf447ee-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47\" (UID: \"5400c3d5-1766-439c-947d-5da15cf447ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:33.044147 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:33.044081    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5400c3d5-1766-439c-947d-5da15cf447ee-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47\" (UID: \"5400c3d5-1766-439c-947d-5da15cf447ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:33.044517 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:33.044480    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5400c3d5-1766-439c-947d-5da15cf447ee-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47\" (UID: \"5400c3d5-1766-439c-947d-5da15cf447ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:33.044594 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:33.044535    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5400c3d5-1766-439c-947d-5da15cf447ee-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47\" (UID: \"5400c3d5-1766-439c-947d-5da15cf447ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:33.044658 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:33.044584    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5400c3d5-1766-439c-947d-5da15cf447ee-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47\" (UID: \"5400c3d5-1766-439c-947d-5da15cf447ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:33.046343 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:33.046321    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5400c3d5-1766-439c-947d-5da15cf447ee-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47\" (UID: \"5400c3d5-1766-439c-947d-5da15cf447ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:33.046615 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:33.046596    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5400c3d5-1766-439c-947d-5da15cf447ee-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47\" (UID: \"5400c3d5-1766-439c-947d-5da15cf447ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:33.052813 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:33.052783    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-298nr\" (UniqueName: \"kubernetes.io/projected/5400c3d5-1766-439c-947d-5da15cf447ee-kube-api-access-298nr\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47\" (UID: \"5400c3d5-1766-439c-947d-5da15cf447ee\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:33.210678 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:33.210635    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:33.482900 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:33.482872    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"]
Sep 14 07:58:33.485487 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:58:33.485456    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5400c3d5_1766_439c_947d_5da15cf447ee.slice/crio-f26972e378d35d631e238e4a8e1e691fa8292359b28f17ed8d182750ed9522cd WatchSource:0}: Error finding container f26972e378d35d631e238e4a8e1e691fa8292359b28f17ed8d182750ed9522cd: Status 404 returned error can't find the container with id f26972e378d35d631e238e4a8e1e691fa8292359b28f17ed8d182750ed9522cd
Sep 14 07:58:33.667858 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:33.667827    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:33.834519 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:33.834422    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47" event={"ID":"5400c3d5-1766-439c-947d-5da15cf447ee","Type":"ContainerStarted","Data":"5655d79740c3b1bdf4e9da768bbe60d03bfcf198e7f2969a2ab0c3aeb4b480c6"}
Sep 14 07:58:33.834519 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:33.834481    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47" event={"ID":"5400c3d5-1766-439c-947d-5da15cf447ee","Type":"ContainerStarted","Data":"f26972e378d35d631e238e4a8e1e691fa8292359b28f17ed8d182750ed9522cd"}
Sep 14 07:58:33.854664 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:33.854612    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47" podStartSLOduration=1.6643348420000001 podStartE2EDuration="1.854594497s" totalImagesPullingTime="190.259655ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:58:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:58:33.853349282 +0000 UTC m=+616.309799275" watchObservedRunningTime="2026-09-14 07:58:33.854594497 +0000 UTC m=+616.311044490"
Sep 14 07:58:34.403602 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:34.403566    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 07:58:43.211226 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:43.211139    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:43.211226 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:43.211217    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:43.224049 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:43.224021    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:43.890638 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:43.890603    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47"
Sep 14 07:58:49.010828 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.010793    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"]
Sep 14 07:58:49.014077 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.014054    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:49.016115 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.016098    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 14 07:58:49.022703 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.022681    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"]
Sep 14 07:58:49.032188 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.032163    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:49.079773 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.079738    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c95803a4-5a40-40e5-bee2-22012d82167f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqcnm\" (UID: \"c95803a4-5a40-40e5-bee2-22012d82167f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:49.079773 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.079777    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c95803a4-5a40-40e5-bee2-22012d82167f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqcnm\" (UID: \"c95803a4-5a40-40e5-bee2-22012d82167f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:49.080036 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.079819    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c95803a4-5a40-40e5-bee2-22012d82167f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqcnm\" (UID: \"c95803a4-5a40-40e5-bee2-22012d82167f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:49.080036 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.079878    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c95803a4-5a40-40e5-bee2-22012d82167f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqcnm\" (UID: \"c95803a4-5a40-40e5-bee2-22012d82167f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:49.080036 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.079944    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c95803a4-5a40-40e5-bee2-22012d82167f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqcnm\" (UID: \"c95803a4-5a40-40e5-bee2-22012d82167f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:49.080036 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.079984    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgrmv\" (UniqueName: \"kubernetes.io/projected/c95803a4-5a40-40e5-bee2-22012d82167f-kube-api-access-fgrmv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqcnm\" (UID: \"c95803a4-5a40-40e5-bee2-22012d82167f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:49.181169 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.181126    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c95803a4-5a40-40e5-bee2-22012d82167f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqcnm\" (UID: \"c95803a4-5a40-40e5-bee2-22012d82167f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:49.181169 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.181171    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c95803a4-5a40-40e5-bee2-22012d82167f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqcnm\" (UID: \"c95803a4-5a40-40e5-bee2-22012d82167f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:49.181423 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.181252    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c95803a4-5a40-40e5-bee2-22012d82167f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqcnm\" (UID: \"c95803a4-5a40-40e5-bee2-22012d82167f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:49.181423 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.181297    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c95803a4-5a40-40e5-bee2-22012d82167f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqcnm\" (UID: \"c95803a4-5a40-40e5-bee2-22012d82167f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:49.181423 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.181354    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c95803a4-5a40-40e5-bee2-22012d82167f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqcnm\" (UID: \"c95803a4-5a40-40e5-bee2-22012d82167f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:49.181423 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.181396    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgrmv\" (UniqueName: \"kubernetes.io/projected/c95803a4-5a40-40e5-bee2-22012d82167f-kube-api-access-fgrmv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqcnm\" (UID: \"c95803a4-5a40-40e5-bee2-22012d82167f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:49.181701 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.181671    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c95803a4-5a40-40e5-bee2-22012d82167f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqcnm\" (UID: \"c95803a4-5a40-40e5-bee2-22012d82167f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:49.181829 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.181800    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c95803a4-5a40-40e5-bee2-22012d82167f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqcnm\" (UID: \"c95803a4-5a40-40e5-bee2-22012d82167f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:49.181954 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.181880    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c95803a4-5a40-40e5-bee2-22012d82167f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqcnm\" (UID: \"c95803a4-5a40-40e5-bee2-22012d82167f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:49.183648 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.183628    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c95803a4-5a40-40e5-bee2-22012d82167f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqcnm\" (UID: \"c95803a4-5a40-40e5-bee2-22012d82167f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:49.183726 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.183709    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c95803a4-5a40-40e5-bee2-22012d82167f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqcnm\" (UID: \"c95803a4-5a40-40e5-bee2-22012d82167f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:49.189832 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.189799    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgrmv\" (UniqueName: \"kubernetes.io/projected/c95803a4-5a40-40e5-bee2-22012d82167f-kube-api-access-fgrmv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-kqcnm\" (UID: \"c95803a4-5a40-40e5-bee2-22012d82167f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:49.326275 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.326182    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:49.599443 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.599408    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"]
Sep 14 07:58:49.605362 ip-10-0-138-28 kubenswrapper[2562]: W0914 07:58:49.605332    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc95803a4_5a40_40e5_bee2_22012d82167f.slice/crio-96b2653aa5afccd9a15145c2929710b7e97dbc3c69610cf54fb45c9bdcde15ff WatchSource:0}: Error finding container 96b2653aa5afccd9a15145c2929710b7e97dbc3c69610cf54fb45c9bdcde15ff: Status 404 returned error can't find the container with id 96b2653aa5afccd9a15145c2929710b7e97dbc3c69610cf54fb45c9bdcde15ff
Sep 14 07:58:49.606643 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.606622    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 07:58:49.909157 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.909113    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm" event={"ID":"c95803a4-5a40-40e5-bee2-22012d82167f","Type":"ContainerStarted","Data":"bf92833512ff670b3ad313bd125033c434b9c9dae0b2a4d1895328eaed8f42a8"}
Sep 14 07:58:49.909346 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.909177    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm" event={"ID":"c95803a4-5a40-40e5-bee2-22012d82167f","Type":"ContainerStarted","Data":"96b2653aa5afccd9a15145c2929710b7e97dbc3c69610cf54fb45c9bdcde15ff"}
Sep 14 07:58:49.927437 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:49.927383    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm" podStartSLOduration=1.74218383 podStartE2EDuration="1.927365469s" totalImagesPullingTime="185.181639ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:58:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:58:49.926254134 +0000 UTC m=+632.382704128" watchObservedRunningTime="2026-09-14 07:58:49.927365469 +0000 UTC m=+632.383815461"
Sep 14 07:58:50.905591 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:50.905553    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 07:58:59.327199 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:59.327155    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:59.327694 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:59.327222    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:59.339994 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:59.339966    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:58:59.928198 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:59.928154    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 07:58:59.967159 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:58:59.967126    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-kqcnm"
Sep 14 07:59:02.458444 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:59:02.458402    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 07:59:14.144172 ip-10-0-138-28 kubenswrapper[2562]: I0914 07:59:14.144121    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:13.616964 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:13.616880    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:13.718124 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:13.718084    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:14.748953 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:14.748918    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:14.850127 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:14.850081    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:30.254189 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.254156    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:30.703522 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.703458    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"]
Sep 14 08:00:30.706955 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.706931    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.710278 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.710252    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-814672-openshift-default-dockercfg-krmck\""
Sep 14 08:00:30.722469 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.722439    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"]
Sep 14 08:00:30.785187 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.785150    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"]
Sep 14 08:00:30.788985 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.788959    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:30.791745 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.791724    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-3a842e-openshift-default-dockercfg-7tv99\""
Sep 14 08:00:30.796088 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.796059    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-workload-certs\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.796226 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.796109    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dgf4\" (UniqueName: \"kubernetes.io/projected/633c40e2-8c81-4de3-b01a-7dfe48f1c292-kube-api-access-4dgf4\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.796714 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.796176    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-envoy\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.796827 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.796762    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istiod-ca-cert\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.796827 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.796803    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-data\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.796956 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.796864    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-workload-socket\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.796956 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.796894    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-token\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.797068 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.796994    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-podinfo\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.797068 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.797042    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-credential-socket\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.805380 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.805350    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"]
Sep 14 08:00:30.898515 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.898464    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-podinfo\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.898696 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.898534    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-token\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:30.898696 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.898564    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-credential-socket\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.898696 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.898581    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-envoy\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:30.898696 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.898607    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-credential-socket\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:30.898696 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.898680    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-workload-certs\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.898928 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.898723    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dgf4\" (UniqueName: \"kubernetes.io/projected/633c40e2-8c81-4de3-b01a-7dfe48f1c292-kube-api-access-4dgf4\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.898928 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.898762    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-workload-socket\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:30.898928 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.898781    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rbws\" (UniqueName: \"kubernetes.io/projected/f0d3cbc9-be52-4983-901e-76e256c6abae-kube-api-access-4rbws\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:30.898928 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.898825    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-envoy\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.898928 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.898883    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-podinfo\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:30.898928 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.898924    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0d3cbc9-be52-4983-901e-76e256c6abae-istiod-ca-cert\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:30.899241 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.898978    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istiod-ca-cert\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.899241 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.898994    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-credential-socket\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.899241 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.899013    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-data\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.899241 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.899029    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-workload-certs\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.899241 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.899050    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-workload-certs\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:30.899241 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.899114    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-workload-socket\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.899241 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.899136    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-token\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.899241 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.899165    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-data\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:30.899759 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.899287    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-data\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.899759 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.899395    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-workload-socket\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.899759 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.899651    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istiod-ca-cert\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.901030 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.901009    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-podinfo\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.901131 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.901060    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-envoy\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.907406 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.907379    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-token\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.907525 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.907477    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dgf4\" (UniqueName: \"kubernetes.io/projected/633c40e2-8c81-4de3-b01a-7dfe48f1c292-kube-api-access-4dgf4\") pod \"e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") " pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:30.914735 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.914702    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:30.999887 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.999799    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-podinfo\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:30.999887 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.999843    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0d3cbc9-be52-4983-901e-76e256c6abae-istiod-ca-cert\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:31.000110 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.999891    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-workload-certs\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:31.000110 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.999931    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-data\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:31.000110 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:30.999989    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-token\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:31.000110 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.000013    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-envoy\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:31.000110 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.000039    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-credential-socket\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:31.000362 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.000113    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-workload-socket\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:31.000362 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.000145    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rbws\" (UniqueName: \"kubernetes.io/projected/f0d3cbc9-be52-4983-901e-76e256c6abae-kube-api-access-4rbws\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:31.000362 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.000344    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-workload-certs\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:31.000672 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.000648    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-credential-socket\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:31.000770 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.000742    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-workload-socket\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:31.000770 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.000752    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0d3cbc9-be52-4983-901e-76e256c6abae-istiod-ca-cert\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:31.000922 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.000901    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-data\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:31.002447 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.002427    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-podinfo\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:31.002860 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.002837    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-envoy\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:31.009801 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.009747    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:31.009801 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.009791    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-token\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:31.010029 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.009970    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rbws\" (UniqueName: \"kubernetes.io/projected/f0d3cbc9-be52-4983-901e-76e256c6abae-kube-api-access-4rbws\") pod \"e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") " pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:31.017694 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.017672    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:31.101207 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.101174    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:31.298367 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.298331    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"]
Sep 14 08:00:31.300858 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:00:31.300828    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod633c40e2_8c81_4de3_b01a_7dfe48f1c292.slice/crio-caf93fe823fde09421f88551e6bc4f530f11a635b55c56bad93be17efe90c514 WatchSource:0}: Error finding container caf93fe823fde09421f88551e6bc4f530f11a635b55c56bad93be17efe90c514: Status 404 returned error can't find the container with id caf93fe823fde09421f88551e6bc4f530f11a635b55c56bad93be17efe90c514
Sep 14 08:00:31.303156 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.303123    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:00:31.303244 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.303199    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:00:31.303244 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.303233    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:00:31.361711 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.361676    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp" event={"ID":"633c40e2-8c81-4de3-b01a-7dfe48f1c292","Type":"ContainerStarted","Data":"caf93fe823fde09421f88551e6bc4f530f11a635b55c56bad93be17efe90c514"}
Sep 14 08:00:31.386281 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.386212    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"]
Sep 14 08:00:31.389947 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:00:31.389903    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0d3cbc9_be52_4983_901e_76e256c6abae.slice/crio-4ee791db6c86228c23f6dae7943e92d0c6aa199503379da59139e297134898c5 WatchSource:0}: Error finding container 4ee791db6c86228c23f6dae7943e92d0c6aa199503379da59139e297134898c5: Status 404 returned error can't find the container with id 4ee791db6c86228c23f6dae7943e92d0c6aa199503379da59139e297134898c5
Sep 14 08:00:31.392387 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.392345    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:00:31.392489 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.392433    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:00:31.392489 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.392471    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:00:31.686572 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.686534    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:31.791078 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:31.791039    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:32.214928 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:32.214898    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:32.368823 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:32.368779    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp" event={"ID":"633c40e2-8c81-4de3-b01a-7dfe48f1c292","Type":"ContainerStarted","Data":"caa481dc1b81fb749f486e6e55ac9620fb0497ea0e98b33c7f9ad2d220f6c138"}
Sep 14 08:00:32.370451 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:32.370427    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq" event={"ID":"f0d3cbc9-be52-4983-901e-76e256c6abae","Type":"ContainerStarted","Data":"d2ca80d73a73a476743524dc1cc539ab59c29bae6d726251b646214bd6ff245b"}
Sep 14 08:00:32.370668 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:32.370463    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq" event={"ID":"f0d3cbc9-be52-4983-901e-76e256c6abae","Type":"ContainerStarted","Data":"4ee791db6c86228c23f6dae7943e92d0c6aa199503379da59139e297134898c5"}
Sep 14 08:00:32.403228 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:32.403164    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp" podStartSLOduration=2.403146195 podStartE2EDuration="2.403146195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:32.402232578 +0000 UTC m=+734.858682604" watchObservedRunningTime="2026-09-14 08:00:32.403146195 +0000 UTC m=+734.859596178"
Sep 14 08:00:32.431335 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:32.431275    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq" podStartSLOduration=2.431259523 podStartE2EDuration="2.431259523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:32.427741923 +0000 UTC m=+734.884191906" watchObservedRunningTime="2026-09-14 08:00:32.431259523 +0000 UTC m=+734.887709516"
Sep 14 08:00:33.017892 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:33.017841    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:33.023156 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:33.023125    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:33.101716 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:33.101674    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:33.106680 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:33.106650    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:33.375326 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:33.375221    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:33.375326 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:33.375278    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:33.376120 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:33.376102    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:33.376188 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:33.376156    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:34.255403 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:34.255373    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:34.793184 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:34.793154    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:00:34.985753 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:34.985723    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:35.083003 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:35.082917    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:35.695016 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:35.694977    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:35.791798 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:35.791755    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:35.887069 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:35.887032    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:36.367639 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:36.367604    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:36.467858 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:36.467822    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:36.839072 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:36.839035    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:36.934056 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:36.934019    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:37.032756 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:37.032718    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:38.157064 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:38.157026    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:38.240250 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:38.240205    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:38.389346 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:38.389312    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:38.714781 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:38.714740    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:38.812903 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:38.812867    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:39.242870 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.242835    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:39.691191 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.691161    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"]
Sep 14 08:00:39.694976 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.694950    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.697074 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.697052    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-22b517-openshift-default-dockercfg-s798p\""
Sep 14 08:00:39.707595 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.707570    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"]
Sep 14 08:00:39.734175 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.734145    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:39.785140 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.785085    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-data\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.785343 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.785207    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istiod-ca-cert\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.785343 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.785247    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nphbg\" (UniqueName: \"kubernetes.io/projected/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-kube-api-access-nphbg\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.785343 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.785289    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-workload-certs\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.785343 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.785312    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-podinfo\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.785609 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.785349    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-workload-socket\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.785609 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.785377    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-token\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.785609 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.785471    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-credential-socket\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.785609 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.785531    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-envoy\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.805121 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.805085    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:39.886403 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.886358    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-workload-certs\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.886403 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.886400    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-podinfo\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.886686 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.886422    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-workload-socket\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.886686 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.886543    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-token\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.886686 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.886635    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-credential-socket\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.886686 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.886671    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-envoy\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.886880 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.886769    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-data\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.886880 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.886808    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istiod-ca-cert\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.886880 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.886850    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nphbg\" (UniqueName: \"kubernetes.io/projected/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-kube-api-access-nphbg\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.887061 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.886851    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-workload-certs\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.887061 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.886969    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-workload-socket\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.887061 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.887027    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-credential-socket\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.887214 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.887177    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-data\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.887384 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.887367    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istiod-ca-cert\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.888832 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.888811    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-envoy\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.888983 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.888965    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-podinfo\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.901183 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.901152    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-token\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.901717 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.901696    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nphbg\" (UniqueName: \"kubernetes.io/projected/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-kube-api-access-nphbg\") pod \"e2e-autores-22b517-openshift-default-7b499f4984-kxs9b\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") " pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:39.920773 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:39.920747    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:40.007753 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.007667    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:40.219876 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.219840    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:40.285012 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.284924    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"]
Sep 14 08:00:40.287435 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.287403    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:40.288490 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:00:40.288459    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod732d49d6_cc23_4fcb_b5ff_7dad71ba89d0.slice/crio-4400356fbf67559a8b962aedaf8b5c31a42ed8d6502ac42bb991df0ae8109195 WatchSource:0}: Error finding container 4400356fbf67559a8b962aedaf8b5c31a42ed8d6502ac42bb991df0ae8109195: Status 404 returned error can't find the container with id 4400356fbf67559a8b962aedaf8b5c31a42ed8d6502ac42bb991df0ae8109195
Sep 14 08:00:40.290676 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.290646    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:00:40.290757 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.290717    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:00:40.290757 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.290744    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:00:40.410481 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.410440    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b" event={"ID":"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0","Type":"ContainerStarted","Data":"0ca7ed0ae57e249787379aad80a05caf93212a1497baedcb6cce8badf3dd9bee"}
Sep 14 08:00:40.410687 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.410521    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b" event={"ID":"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0","Type":"ContainerStarted","Data":"4400356fbf67559a8b962aedaf8b5c31a42ed8d6502ac42bb991df0ae8109195"}
Sep 14 08:00:40.432696 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.432626    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b" podStartSLOduration=1.432601192 podStartE2EDuration="1.432601192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:40.430605062 +0000 UTC m=+742.887055061" watchObservedRunningTime="2026-09-14 08:00:40.432601192 +0000 UTC m=+742.889051187"
Sep 14 08:00:40.453350 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.453302    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:40.773992 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.773955    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"]
Sep 14 08:00:40.778003 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.777979    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.790224 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.790203    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a9d21f4e-openshift-default-dockercfg-8wvwq\""
Sep 14 08:00:40.798963 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.798939    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:40.811995 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.811960    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"]
Sep 14 08:00:40.895886 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.895847    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/110b1602-7bad-4bd0-8175-54982ec82282-istiod-ca-cert\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.896074 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.895890    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/110b1602-7bad-4bd0-8175-54982ec82282-istio-podinfo\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.896074 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.896028    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/110b1602-7bad-4bd0-8175-54982ec82282-istio-token\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.896200 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.896072    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dnzb\" (UniqueName: \"kubernetes.io/projected/110b1602-7bad-4bd0-8175-54982ec82282-kube-api-access-2dnzb\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.896200 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.896137    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-istio-data\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.896200 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.896168    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-credential-socket\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.896359 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.896278    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-workload-socket\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.896359 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.896313    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-istio-envoy\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.896359 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.896337    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-workload-certs\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.997143 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.997104    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/110b1602-7bad-4bd0-8175-54982ec82282-istio-podinfo\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.997309 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.997171    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/110b1602-7bad-4bd0-8175-54982ec82282-istio-token\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.997309 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.997190    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dnzb\" (UniqueName: \"kubernetes.io/projected/110b1602-7bad-4bd0-8175-54982ec82282-kube-api-access-2dnzb\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.997309 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.997215    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-istio-data\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.997309 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.997234    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-credential-socket\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.997309 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.997283    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-workload-socket\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.997608 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.997397    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-istio-envoy\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.997608 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.997432    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-workload-certs\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.997608 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.997482    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/110b1602-7bad-4bd0-8175-54982ec82282-istiod-ca-cert\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.997817 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.997793    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-workload-socket\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.998180 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.997830    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-credential-socket\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.998180 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.997994    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-istio-data\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.998180 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.998019    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-workload-certs\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.998309 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.998227    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/110b1602-7bad-4bd0-8175-54982ec82282-istiod-ca-cert\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.999706 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.999677    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-istio-envoy\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:40.999791 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:40.999752    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/110b1602-7bad-4bd0-8175-54982ec82282-istio-podinfo\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:41.007981 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:41.007958    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:41.008057 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:41.008003    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/110b1602-7bad-4bd0-8175-54982ec82282-istio-token\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:41.008418 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:41.008397    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dnzb\" (UniqueName: \"kubernetes.io/projected/110b1602-7bad-4bd0-8175-54982ec82282-kube-api-access-2dnzb\") pod \"e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") " pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:41.013189 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:41.013166    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:41.081111 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:41.081029    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:41.088986 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:41.088955    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:41.381771 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:41.381744    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"]
Sep 14 08:00:41.383792 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:00:41.383765    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod110b1602_7bad_4bd0_8175_54982ec82282.slice/crio-e8e8c484c0bd28839289a3a33f50226a79f9ed8597953e899ba2c1870a5271f9 WatchSource:0}: Error finding container e8e8c484c0bd28839289a3a33f50226a79f9ed8597953e899ba2c1870a5271f9: Status 404 returned error can't find the container with id e8e8c484c0bd28839289a3a33f50226a79f9ed8597953e899ba2c1870a5271f9
Sep 14 08:00:41.386092 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:41.386058    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:00:41.386182 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:41.386125    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:00:41.386182 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:41.386155    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:00:41.416509 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:41.416471    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv" event={"ID":"110b1602-7bad-4bd0-8175-54982ec82282","Type":"ContainerStarted","Data":"e8e8c484c0bd28839289a3a33f50226a79f9ed8597953e899ba2c1870a5271f9"}
Sep 14 08:00:41.416796 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:41.416778    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:41.417840 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:41.417822    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:00:41.850084 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:41.850047    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:42.422856 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:42.422814    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv" event={"ID":"110b1602-7bad-4bd0-8175-54982ec82282","Type":"ContainerStarted","Data":"a811a850e6b5c399170341316b091d823b0d1b940a26db5b02978159aa625cb8"}
Sep 14 08:00:42.448680 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:42.448627    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv" podStartSLOduration=2.448611306 podStartE2EDuration="2.448611306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:42.446587813 +0000 UTC m=+744.903037808" watchObservedRunningTime="2026-09-14 08:00:42.448611306 +0000 UTC m=+744.905061363"
Sep 14 08:00:42.628594 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:42.628558    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:42.730757 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:42.730702    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:42.829620 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:42.829587    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:43.089375 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:43.089284    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:43.094268 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:43.094239    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:43.428761 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:43.428673    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:43.429549 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:43.429527    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:00:44.008003 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:44.007963    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:44.142735 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:44.142697    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:44.442866 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:44.442838    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:44.910804 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:44.910714    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:45.019021 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:45.018983    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:45.554562 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:45.554525    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:46.794288 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:46.794238    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:47.572369 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:47.572335    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:48.459874 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:48.459840    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:48.555474 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:48.555429    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:48.646164 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:48.646127    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:48.884409 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:48.884269    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:49.012184 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:49.012152    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:49.110838 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:49.110806    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:49.194150 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:49.194117    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:49.332172 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:49.332142    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:49.455730 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:49.455615    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:49.550573 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:49.550541    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:49.705292 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:49.705259    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:49.799023 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:49.798932    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:50.408775 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:50.408715    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:51.102569 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:51.102458    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:51.202645 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:51.202604    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:51.548182 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:51.548144    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:51.694561 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:51.694515    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:51.802440 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:51.802334    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:51.921866 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:51.921816    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:52.021813 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:52.021764    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:53.454719 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:53.454674    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:53.636836 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:53.636798    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:54.340408 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:54.340377    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:54.412615 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:54.412529    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:00:54.513225 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:54.513140    2562 generic.go:356] "Generic (PLEG): container finished" podID="633c40e2-8c81-4de3-b01a-7dfe48f1c292" containerID="caa481dc1b81fb749f486e6e55ac9620fb0497ea0e98b33c7f9ad2d220f6c138" exitCode=0
Sep 14 08:00:54.513225 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:54.513208    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp" event={"ID":"633c40e2-8c81-4de3-b01a-7dfe48f1c292","Type":"ContainerDied","Data":"caa481dc1b81fb749f486e6e55ac9620fb0497ea0e98b33c7f9ad2d220f6c138"}
Sep 14 08:00:54.513744 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:54.513683    2562 scope.go:117] "RemoveContainer" containerID="caa481dc1b81fb749f486e6e55ac9620fb0497ea0e98b33c7f9ad2d220f6c138"
Sep 14 08:00:54.514998 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:54.514966    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:00:54.515089 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:54.515048    2562 generic.go:356] "Generic (PLEG): container finished" podID="f0d3cbc9-be52-4983-901e-76e256c6abae" containerID="d2ca80d73a73a476743524dc1cc539ab59c29bae6d726251b646214bd6ff245b" exitCode=0
Sep 14 08:00:54.515089 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:54.515070    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq" event={"ID":"f0d3cbc9-be52-4983-901e-76e256c6abae","Type":"ContainerDied","Data":"d2ca80d73a73a476743524dc1cc539ab59c29bae6d726251b646214bd6ff245b"}
Sep 14 08:00:54.515163 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:54.515051    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:00:54.515197 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:54.515176    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:00:54.515478 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:54.515455    2562 scope.go:117] "RemoveContainer" containerID="d2ca80d73a73a476743524dc1cc539ab59c29bae6d726251b646214bd6ff245b"
Sep 14 08:00:54.516547 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:54.516472    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:00:54.516647 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:54.516575    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:00:54.516647 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:54.516637    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:00:54.570864 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:54.570782    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:54.699811 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:54.699777    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:54.801927 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:54.801803    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:55.047440 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.046922    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"]
Sep 14 08:00:55.066586 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.065187    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"]
Sep 14 08:00:55.066586 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.065310    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.067884 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.067859    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-5ae4e6-openshift-default-dockercfg-5qn2v\""
Sep 14 08:00:55.082595 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.082123    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:55.171100 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.170173    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc26p\" (UniqueName: \"kubernetes.io/projected/8512cd08-a696-4695-9bbf-c6c3d68d7329-kube-api-access-vc26p\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.171100 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.170381    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8512cd08-a696-4695-9bbf-c6c3d68d7329-istiod-ca-cert\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.171100 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.170481    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-workload-socket\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.171100 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.170567    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-podinfo\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.171100 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.170680    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-data\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.171100 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.170713    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-envoy\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.171100 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.170763    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-credential-socket\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.171100 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.170847    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-token\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.171100 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.170891    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-workload-certs\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.272467 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.272427    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-workload-socket\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.272658 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.272489    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-podinfo\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.272658 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.272575    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-data\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.272658 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.272607    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-envoy\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.272658 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.272644    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-credential-socket\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.272795 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.272672    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-token\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.272795 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.272708    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-workload-certs\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.272795 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.272765    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc26p\" (UniqueName: \"kubernetes.io/projected/8512cd08-a696-4695-9bbf-c6c3d68d7329-kube-api-access-vc26p\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.272887 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.272805    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8512cd08-a696-4695-9bbf-c6c3d68d7329-istiod-ca-cert\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.273650 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.273621    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-workload-socket\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.273815 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.273666    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8512cd08-a696-4695-9bbf-c6c3d68d7329-istiod-ca-cert\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.273905 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.273703    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-credential-socket\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.273987 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.273751    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-data\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.274055 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.273938    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-workload-certs\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.275429 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.275391    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-envoy\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.275558 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.275536    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-podinfo\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.284773 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.284750    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-token\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.299075 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.299050    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:55.320472 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.320411    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc26p\" (UniqueName: \"kubernetes.io/projected/8512cd08-a696-4695-9bbf-c6c3d68d7329-kube-api-access-vc26p\") pod \"e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") " pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.385143 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.385096    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:55.523496 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.523442    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq" event={"ID":"f0d3cbc9-be52-4983-901e-76e256c6abae","Type":"ContainerStarted","Data":"987b61705e37f5fdff6f2cbd5d02cf18541ed5145501dcd6e1f33b6d20c4f7e4"}
Sep 14 08:00:55.525528 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.525484    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp" event={"ID":"633c40e2-8c81-4de3-b01a-7dfe48f1c292","Type":"ContainerStarted","Data":"46ec19b308308b5b4b05ddb361566a5b7f0833f86b64549b2a04f9db05df8892"}
Sep 14 08:00:55.698559 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.698534    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"]
Sep 14 08:00:55.700696 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:00:55.700666    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8512cd08_a696_4695_9bbf_c6c3d68d7329.slice/crio-948433736fbcc19dc41fdeb1a778c3131008a9e08f74b229953ca47a04a86df2 WatchSource:0}: Error finding container 948433736fbcc19dc41fdeb1a778c3131008a9e08f74b229953ca47a04a86df2: Status 404 returned error can't find the container with id 948433736fbcc19dc41fdeb1a778c3131008a9e08f74b229953ca47a04a86df2
Sep 14 08:00:55.703119 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.703084    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:00:55.703238 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.703167    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:00:55.703238 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.703211    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:00:55.949607 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:55.949494    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:56.018829 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:56.018789    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:00:56.019022 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:56.018858    2562 patch_prober.go:28] interesting pod/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 14 08:00:56.019022 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:56.018926    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp" podUID="633c40e2-8c81-4de3-b01a-7dfe48f1c292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 14 08:00:56.102536 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:56.102480    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:00:56.102741 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:56.102642    2562 patch_prober.go:28] interesting pod/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 14 08:00:56.102741 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:56.102689    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq" podUID="f0d3cbc9-be52-4983-901e-76e256c6abae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 14 08:00:56.545524 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:56.545166    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt" event={"ID":"8512cd08-a696-4695-9bbf-c6c3d68d7329","Type":"ContainerStarted","Data":"891e088b638cc13f7d54787c978c0b4a218e3fd2e2ce6c3df589a7e2ad74c0b9"}
Sep 14 08:00:56.545524 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:56.545253    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt" event={"ID":"8512cd08-a696-4695-9bbf-c6c3d68d7329","Type":"ContainerStarted","Data":"948433736fbcc19dc41fdeb1a778c3131008a9e08f74b229953ca47a04a86df2"}
Sep 14 08:00:57.027524 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:57.019924    2562 patch_prober.go:28] interesting pod/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 14 08:00:57.027524 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:57.019999    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp" podUID="633c40e2-8c81-4de3-b01a-7dfe48f1c292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 14 08:00:57.114525 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:57.112940    2562 patch_prober.go:28] interesting pod/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 14 08:00:57.114525 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:57.113017    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq" podUID="f0d3cbc9-be52-4983-901e-76e256c6abae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 14 08:00:57.228029 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:57.227991    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:57.345443 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:57.345348    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:57.388454 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:57.388414    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:57.396807 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:57.396775    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:57.419625 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:57.419557    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt" podStartSLOduration=2.419533354 podStartE2EDuration="2.419533354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:56.576638359 +0000 UTC m=+759.033088398" watchObservedRunningTime="2026-09-14 08:00:57.419533354 +0000 UTC m=+759.875983349"
Sep 14 08:00:57.565054 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:57.565021    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:57.578525 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:57.575821    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:00:58.019198 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:58.019154    2562 patch_prober.go:28] interesting pod/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 14 08:00:58.019554 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:58.019490    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp" podUID="633c40e2-8c81-4de3-b01a-7dfe48f1c292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 14 08:00:58.031625 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:58.031578    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:58.101946 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:58.101905    2562 patch_prober.go:28] interesting pod/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 14 08:00:58.102248 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:58.102216    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq" podUID="f0d3cbc9-be52-4983-901e-76e256c6abae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 14 08:00:58.135666 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:58.135627    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:58.599490 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:58.599442    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:58.696268 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:58.696237    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:00:59.018878 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:59.018837    2562 patch_prober.go:28] interesting pod/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 14 08:00:59.019190 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:59.019155    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp" podUID="633c40e2-8c81-4de3-b01a-7dfe48f1c292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 14 08:00:59.102407 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:59.102347    2562 patch_prober.go:28] interesting pod/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 14 08:00:59.102629 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:59.102436    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq" podUID="f0d3cbc9-be52-4983-901e-76e256c6abae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 14 08:00:59.222493 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:00:59.222451    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:00.018535 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:00.018472    2562 patch_prober.go:28] interesting pod/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 14 08:01:00.019133 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:00.019098    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp" podUID="633c40e2-8c81-4de3-b01a-7dfe48f1c292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 14 08:01:00.079624 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:00.079577    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:00.102601 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:00.102563    2562 patch_prober.go:28] interesting pod/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 14 08:01:00.102900 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:00.102869    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq" podUID="f0d3cbc9-be52-4983-901e-76e256c6abae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 14 08:01:01.009287 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:01.009252    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:01.018589 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:01.018126    2562 patch_prober.go:28] interesting pod/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 14 08:01:01.018589 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:01.018193    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp" podUID="633c40e2-8c81-4de3-b01a-7dfe48f1c292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 14 08:01:01.108020 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:01.107979    2562 patch_prober.go:28] interesting pod/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 14 08:01:01.108328 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:01.108296    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq" podUID="f0d3cbc9-be52-4983-901e-76e256c6abae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 14 08:01:01.848760 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:01.848716    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:02.018605 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:02.018568    2562 patch_prober.go:28] interesting pod/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body=
Sep 14 08:01:02.019098 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:02.018627    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp" podUID="633c40e2-8c81-4de3-b01a-7dfe48f1c292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused"
Sep 14 08:01:02.105966 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:02.105849    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:01:02.106196 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:02.106173    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:01:02.114344 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:02.114309    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:01:02.854625 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:02.854587    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:02.955168 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:02.955114    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:03.023828 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:03.023791    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:01:03.024300 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:03.024198    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:01:03.025174 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:03.025147    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:01:04.238331 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:04.238291    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:05.603813 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:05.603777    2562 generic.go:356] "Generic (PLEG): container finished" podID="732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" containerID="0ca7ed0ae57e249787379aad80a05caf93212a1497baedcb6cce8badf3dd9bee" exitCode=0
Sep 14 08:01:05.604327 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:05.603861    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b" event={"ID":"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0","Type":"ContainerDied","Data":"0ca7ed0ae57e249787379aad80a05caf93212a1497baedcb6cce8badf3dd9bee"}
Sep 14 08:01:05.604452 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:05.604433    2562 scope.go:117] "RemoveContainer" containerID="0ca7ed0ae57e249787379aad80a05caf93212a1497baedcb6cce8badf3dd9bee"
Sep 14 08:01:05.605864 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:05.605832    2562 generic.go:356] "Generic (PLEG): container finished" podID="110b1602-7bad-4bd0-8175-54982ec82282" containerID="a811a850e6b5c399170341316b091d823b0d1b940a26db5b02978159aa625cb8" exitCode=0
Sep 14 08:01:05.605982 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:05.605896    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv" event={"ID":"110b1602-7bad-4bd0-8175-54982ec82282","Type":"ContainerDied","Data":"a811a850e6b5c399170341316b091d823b0d1b940a26db5b02978159aa625cb8"}
Sep 14 08:01:05.606033 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:05.605996    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:01:05.606108 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:05.606088    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:01:05.606166 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:05.606150    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:01:05.606297 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:05.606282    2562 scope.go:117] "RemoveContainer" containerID="a811a850e6b5c399170341316b091d823b0d1b940a26db5b02978159aa625cb8"
Sep 14 08:01:05.607525 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:05.607478    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:01:05.607620 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:05.607572    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:01:05.607680 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:05.607616    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:01:05.627985 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:05.627920    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:05.734870 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:05.734836    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:05.967160 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:05.967120    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:06.278217 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:06.278078    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:06.368411 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:06.368368    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:06.614468 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:06.614381    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv" event={"ID":"110b1602-7bad-4bd0-8175-54982ec82282","Type":"ContainerStarted","Data":"ce9bed3bd758b5e8d6eda227f56414409ff97c4c70303060fa9ca5f92a2ee7ab"}
Sep 14 08:01:06.616464 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:06.616444    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b" event={"ID":"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0","Type":"ContainerStarted","Data":"d8a306ba1088c317f0132f3d138440d6b9695b79b15003720cc5966d41bd98a4"}
Sep 14 08:01:06.748414 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:06.748371    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:07.008490 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:07.008454    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:01:07.009380 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:07.009353    2562 patch_prober.go:28] interesting pod/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 14 08:01:07.009484 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:07.009410    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b" podUID="732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 14 08:01:07.089728 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:07.089687    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:01:07.090015 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:07.089987    2562 patch_prober.go:28] interesting pod/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Sep 14 08:01:07.090096 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:07.090067    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv" podUID="110b1602-7bad-4bd0-8175-54982ec82282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Sep 14 08:01:07.782340 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:07.782295    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:08.008520 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:08.008478    2562 patch_prober.go:28] interesting pod/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 14 08:01:08.008711 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:08.008556    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b" podUID="732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 14 08:01:08.090291 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:08.090196    2562 patch_prober.go:28] interesting pod/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Sep 14 08:01:08.090291 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:08.090281    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv" podUID="110b1602-7bad-4bd0-8175-54982ec82282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Sep 14 08:01:08.592851 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:08.592810    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:09.008939 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:09.008900    2562 patch_prober.go:28] interesting pod/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 14 08:01:09.009337 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:09.008958    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b" podUID="732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 14 08:01:09.090228 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:09.090183    2562 patch_prober.go:28] interesting pod/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Sep 14 08:01:09.090414 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:09.090255    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv" podUID="110b1602-7bad-4bd0-8175-54982ec82282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Sep 14 08:01:09.387852 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:09.387700    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:09.833395 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:09.833353    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:09.922144 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:09.922111    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:10.008465 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:10.008430    2562 patch_prober.go:28] interesting pod/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 14 08:01:10.008658 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:10.008480    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b" podUID="732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 14 08:01:10.089954 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:10.089872    2562 patch_prober.go:28] interesting pod/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Sep 14 08:01:10.090359 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:10.090334    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv" podUID="110b1602-7bad-4bd0-8175-54982ec82282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Sep 14 08:01:10.253878 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:10.253832    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:01:10.794667 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:10.794629    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:11.008277 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:11.008243    2562 patch_prober.go:28] interesting pod/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Sep 14 08:01:11.008444 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:11.008316    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b" podUID="732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Sep 14 08:01:11.090079 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:11.089974    2562 patch_prober.go:28] interesting pod/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Sep 14 08:01:11.090079 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:11.090042    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv" podUID="110b1602-7bad-4bd0-8175-54982ec82282" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Sep 14 08:01:11.416941 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:11.416864    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:01:11.698443 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:11.698391    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:12.012340 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:12.012257    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:01:12.013355 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:12.013331    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:01:12.094028 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:12.093993    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:01:12.094485 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:12.094469    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:01:12.095329 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:12.095309    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:01:12.657905 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:12.657841    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:13.455337 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:13.455296    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:15.156340 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:15.156306    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:15.262452 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:15.262418    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:15.408094 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:15.407985    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:15.417110 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:15.417077    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"]
Sep 14 08:01:15.417310 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:15.417286    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv" podUID="110b1602-7bad-4bd0-8175-54982ec82282" containerName="istio-proxy" containerID="cri-o://ce9bed3bd758b5e8d6eda227f56414409ff97c4c70303060fa9ca5f92a2ee7ab" gracePeriod=30
Sep 14 08:01:15.504696 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:15.504660    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:15.821286 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:15.821253    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:15.925886 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:15.925854    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:16.023857 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.023824    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:16.122778 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.122684    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:16.310587 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.310552    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:16.387992 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.387922    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"]
Sep 14 08:01:16.390712 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.390691    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.392839 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.392817    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-71665efb-gw-openshift-default-dockercfg-lnml6\""
Sep 14 08:01:16.404805 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.404779    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"]
Sep 14 08:01:16.409700 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.409672    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:16.469153 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.469115    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:16.480644 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.480595    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-workload-certs\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.480644 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.480646    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-envoy\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.480919 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.480685    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istiod-ca-cert\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.480919 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.480762    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-token\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.480919 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.480787    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9hcn\" (UniqueName: \"kubernetes.io/projected/7e2e34e5-5170-4e7c-803b-7940b1e12a09-kube-api-access-b9hcn\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.480919 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.480839    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-credential-socket\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.480919 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.480892    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-podinfo\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.481184 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.480928    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-workload-socket\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.481184 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.480959    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-data\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.581443 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.581401    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-token\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.581443 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.581446    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9hcn\" (UniqueName: \"kubernetes.io/projected/7e2e34e5-5170-4e7c-803b-7940b1e12a09-kube-api-access-b9hcn\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.581707 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.581495    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-credential-socket\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.581707 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.581560    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-podinfo\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.581820 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.581713    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-workload-socket\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.581820 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.581764    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-data\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.581926 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.581830    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-workload-certs\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.581926 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.581864    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-envoy\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.581926 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.581903    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istiod-ca-cert\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.582783 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.582715    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-data\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.582783 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.582733    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-workload-certs\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.582958 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.582812    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-workload-socket\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.582958 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.582832    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-credential-socket\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.582958 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.582894    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istiod-ca-cert\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.584485 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.584459    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-podinfo\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.584663 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.584640    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-envoy\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.593615 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.593587    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-token\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.593822 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.593804    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9hcn\" (UniqueName: \"kubernetes.io/projected/7e2e34e5-5170-4e7c-803b-7940b1e12a09-kube-api-access-b9hcn\") pod \"e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") " pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.701386 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.701351    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:16.862524 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.861513    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:16.958107 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:16.958006    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:17.006570 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:17.006543    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"]
Sep 14 08:01:17.008592 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:01:17.008556    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e2e34e5_5170_4e7c_803b_7940b1e12a09.slice/crio-7a89ec100d2e20d876e546781fb01a1df2da873151413ad2f93b1409e34c5265 WatchSource:0}: Error finding container 7a89ec100d2e20d876e546781fb01a1df2da873151413ad2f93b1409e34c5265: Status 404 returned error can't find the container with id 7a89ec100d2e20d876e546781fb01a1df2da873151413ad2f93b1409e34c5265
Sep 14 08:01:17.010990 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:17.010955    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:01:17.011086 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:17.011023    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:01:17.011086 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:17.011068    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:01:17.552203 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:17.552163    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:17.681058 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:17.681018    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx" event={"ID":"7e2e34e5-5170-4e7c-803b-7940b1e12a09","Type":"ContainerStarted","Data":"f386eea92af836bec1ff5416df087ef07fadc1a0f03adfd731423f6b1ee0b2f7"}
Sep 14 08:01:17.681242 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:17.681075    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx" event={"ID":"7e2e34e5-5170-4e7c-803b-7940b1e12a09","Type":"ContainerStarted","Data":"7a89ec100d2e20d876e546781fb01a1df2da873151413ad2f93b1409e34c5265"}
Sep 14 08:01:17.706523 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:17.701712    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:17.706523 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:17.706330    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx" podStartSLOduration=1.706311071 podStartE2EDuration="1.706311071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:01:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:01:17.702600722 +0000 UTC m=+780.159050710" watchObservedRunningTime="2026-09-14 08:01:17.706311071 +0000 UTC m=+780.162761066"
Sep 14 08:01:18.614537 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:18.614479    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:18.708237 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:18.708201    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:19.269038 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:19.269003    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:19.369469 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:19.369426    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:19.473888 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:19.473855    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:19.691193 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:19.691167    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:19.692312 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:19.692294    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:20.428479 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.428439    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:20.686531 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.686487    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:01:20.697023 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.696996    2562 generic.go:356] "Generic (PLEG): container finished" podID="110b1602-7bad-4bd0-8175-54982ec82282" containerID="ce9bed3bd758b5e8d6eda227f56414409ff97c4c70303060fa9ca5f92a2ee7ab" exitCode=0
Sep 14 08:01:20.697363 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.697046    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"
Sep 14 08:01:20.697363 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.697077    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv" event={"ID":"110b1602-7bad-4bd0-8175-54982ec82282","Type":"ContainerDied","Data":"ce9bed3bd758b5e8d6eda227f56414409ff97c4c70303060fa9ca5f92a2ee7ab"}
Sep 14 08:01:20.697363 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.697132    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv" event={"ID":"110b1602-7bad-4bd0-8175-54982ec82282","Type":"ContainerDied","Data":"e8e8c484c0bd28839289a3a33f50226a79f9ed8597953e899ba2c1870a5271f9"}
Sep 14 08:01:20.697363 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.697163    2562 scope.go:117] "RemoveContainer" containerID="ce9bed3bd758b5e8d6eda227f56414409ff97c4c70303060fa9ca5f92a2ee7ab"
Sep 14 08:01:20.706435 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.706415    2562 scope.go:117] "RemoveContainer" containerID="a811a850e6b5c399170341316b091d823b0d1b940a26db5b02978159aa625cb8"
Sep 14 08:01:20.716536 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.716495    2562 scope.go:117] "RemoveContainer" containerID="ce9bed3bd758b5e8d6eda227f56414409ff97c4c70303060fa9ca5f92a2ee7ab"
Sep 14 08:01:20.716925 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:01:20.716900    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce9bed3bd758b5e8d6eda227f56414409ff97c4c70303060fa9ca5f92a2ee7ab\": container with ID starting with ce9bed3bd758b5e8d6eda227f56414409ff97c4c70303060fa9ca5f92a2ee7ab not found: ID does not exist" containerID="ce9bed3bd758b5e8d6eda227f56414409ff97c4c70303060fa9ca5f92a2ee7ab"
Sep 14 08:01:20.717037 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.716930    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce9bed3bd758b5e8d6eda227f56414409ff97c4c70303060fa9ca5f92a2ee7ab"} err="failed to get container status \"ce9bed3bd758b5e8d6eda227f56414409ff97c4c70303060fa9ca5f92a2ee7ab\": rpc error: code = NotFound desc = could not find container \"ce9bed3bd758b5e8d6eda227f56414409ff97c4c70303060fa9ca5f92a2ee7ab\": container with ID starting with ce9bed3bd758b5e8d6eda227f56414409ff97c4c70303060fa9ca5f92a2ee7ab not found: ID does not exist"
Sep 14 08:01:20.717037 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.716947    2562 scope.go:117] "RemoveContainer" containerID="a811a850e6b5c399170341316b091d823b0d1b940a26db5b02978159aa625cb8"
Sep 14 08:01:20.717272 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:01:20.717254    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a811a850e6b5c399170341316b091d823b0d1b940a26db5b02978159aa625cb8\": container with ID starting with a811a850e6b5c399170341316b091d823b0d1b940a26db5b02978159aa625cb8 not found: ID does not exist" containerID="a811a850e6b5c399170341316b091d823b0d1b940a26db5b02978159aa625cb8"
Sep 14 08:01:20.717349 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.717280    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a811a850e6b5c399170341316b091d823b0d1b940a26db5b02978159aa625cb8"} err="failed to get container status \"a811a850e6b5c399170341316b091d823b0d1b940a26db5b02978159aa625cb8\": rpc error: code = NotFound desc = could not find container \"a811a850e6b5c399170341316b091d823b0d1b940a26db5b02978159aa625cb8\": container with ID starting with a811a850e6b5c399170341316b091d823b0d1b940a26db5b02978159aa625cb8 not found: ID does not exist"
Sep 14 08:01:20.722641 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.722615    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-credential-socket\") pod \"110b1602-7bad-4bd0-8175-54982ec82282\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") "
Sep 14 08:01:20.722758 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.722693    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-workload-certs\") pod \"110b1602-7bad-4bd0-8175-54982ec82282\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") "
Sep 14 08:01:20.722829 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.722804    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/110b1602-7bad-4bd0-8175-54982ec82282-istiod-ca-cert\") pod \"110b1602-7bad-4bd0-8175-54982ec82282\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") "
Sep 14 08:01:20.722889 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.722843    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-workload-socket\") pod \"110b1602-7bad-4bd0-8175-54982ec82282\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") "
Sep 14 08:01:20.722889 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.722874    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/110b1602-7bad-4bd0-8175-54982ec82282-istio-podinfo\") pod \"110b1602-7bad-4bd0-8175-54982ec82282\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") "
Sep 14 08:01:20.722995 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.722904    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-istio-envoy\") pod \"110b1602-7bad-4bd0-8175-54982ec82282\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") "
Sep 14 08:01:20.722995 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.722917    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "110b1602-7bad-4bd0-8175-54982ec82282" (UID: "110b1602-7bad-4bd0-8175-54982ec82282"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:01:20.722995 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.722928    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-istio-data\") pod \"110b1602-7bad-4bd0-8175-54982ec82282\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") "
Sep 14 08:01:20.723156 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.723019    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "110b1602-7bad-4bd0-8175-54982ec82282" (UID: "110b1602-7bad-4bd0-8175-54982ec82282"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:01:20.723156 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.723014    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/110b1602-7bad-4bd0-8175-54982ec82282-istio-token\") pod \"110b1602-7bad-4bd0-8175-54982ec82282\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") "
Sep 14 08:01:20.723156 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.723118    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dnzb\" (UniqueName: \"kubernetes.io/projected/110b1602-7bad-4bd0-8175-54982ec82282-kube-api-access-2dnzb\") pod \"110b1602-7bad-4bd0-8175-54982ec82282\" (UID: \"110b1602-7bad-4bd0-8175-54982ec82282\") "
Sep 14 08:01:20.723304 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.723172    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-istio-data" (OuterVolumeSpecName: "istio-data") pod "110b1602-7bad-4bd0-8175-54982ec82282" (UID: "110b1602-7bad-4bd0-8175-54982ec82282"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:01:20.723304 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.723197    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/110b1602-7bad-4bd0-8175-54982ec82282-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "110b1602-7bad-4bd0-8175-54982ec82282" (UID: "110b1602-7bad-4bd0-8175-54982ec82282"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:01:20.723943 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.723550    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "110b1602-7bad-4bd0-8175-54982ec82282" (UID: "110b1602-7bad-4bd0-8175-54982ec82282"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:01:20.723943 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.723844    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-istio-data\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:01:20.723943 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.723868    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-credential-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:01:20.723943 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.723892    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-workload-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:01:20.723943 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.723909    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/110b1602-7bad-4bd0-8175-54982ec82282-istiod-ca-cert\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:01:20.725876 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.725841    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/110b1602-7bad-4bd0-8175-54982ec82282-istio-token" (OuterVolumeSpecName: "istio-token") pod "110b1602-7bad-4bd0-8175-54982ec82282" (UID: "110b1602-7bad-4bd0-8175-54982ec82282"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:01:20.725968 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.725911    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "110b1602-7bad-4bd0-8175-54982ec82282" (UID: "110b1602-7bad-4bd0-8175-54982ec82282"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:01:20.725968 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.725918    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/110b1602-7bad-4bd0-8175-54982ec82282-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "110b1602-7bad-4bd0-8175-54982ec82282" (UID: "110b1602-7bad-4bd0-8175-54982ec82282"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:01:20.726337 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.726317    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/110b1602-7bad-4bd0-8175-54982ec82282-kube-api-access-2dnzb" (OuterVolumeSpecName: "kube-api-access-2dnzb") pod "110b1602-7bad-4bd0-8175-54982ec82282" (UID: "110b1602-7bad-4bd0-8175-54982ec82282"). InnerVolumeSpecName "kube-api-access-2dnzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:01:20.824861 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.824825    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-workload-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:01:20.824861 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.824854    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/110b1602-7bad-4bd0-8175-54982ec82282-istio-podinfo\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:01:20.824861 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.824863    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/110b1602-7bad-4bd0-8175-54982ec82282-istio-envoy\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:01:20.824861 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.824872    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/110b1602-7bad-4bd0-8175-54982ec82282-istio-token\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:01:20.824861 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:20.824882    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dnzb\" (UniqueName: \"kubernetes.io/projected/110b1602-7bad-4bd0-8175-54982ec82282-kube-api-access-2dnzb\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:01:21.018979 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:21.018947    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"]
Sep 14 08:01:21.027729 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:21.027697    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a9d21f4e-openshift-default-b967cf8b8-wnbwv"]
Sep 14 08:01:21.500034 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:21.500000    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:21.602679 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:21.602648    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:21.704962 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:21.704877    2562 generic.go:356] "Generic (PLEG): container finished" podID="8512cd08-a696-4695-9bbf-c6c3d68d7329" containerID="891e088b638cc13f7d54787c978c0b4a218e3fd2e2ce6c3df589a7e2ad74c0b9" exitCode=0
Sep 14 08:01:21.704962 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:21.704953    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt" event={"ID":"8512cd08-a696-4695-9bbf-c6c3d68d7329","Type":"ContainerDied","Data":"891e088b638cc13f7d54787c978c0b4a218e3fd2e2ce6c3df589a7e2ad74c0b9"}
Sep 14 08:01:21.705465 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:21.705451    2562 scope.go:117] "RemoveContainer" containerID="891e088b638cc13f7d54787c978c0b4a218e3fd2e2ce6c3df589a7e2ad74c0b9"
Sep 14 08:01:21.706596 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:21.706569    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:01:21.706664 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:21.706635    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:01:21.706701 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:21.706669    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:01:21.815288 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:21.815231    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:22.064446 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:22.064354    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:22.113105 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:22.113075    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="110b1602-7bad-4bd0-8175-54982ec82282" path="/var/lib/kubelet/pods/110b1602-7bad-4bd0-8175-54982ec82282/volumes"
Sep 14 08:01:22.164286 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:22.164246    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:22.544402 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:22.544356    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:22.712242 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:22.712209    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt" event={"ID":"8512cd08-a696-4695-9bbf-c6c3d68d7329","Type":"ContainerStarted","Data":"d17232d3dd3ad19f8744ad693a56d330898dabec149cd26fd250c1621a12f15b"}
Sep 14 08:01:23.386439 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:23.386395    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:01:23.387703 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:23.387675    2562 patch_prober.go:28] interesting pod/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 14 08:01:23.387770 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:23.387735    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt" podUID="8512cd08-a696-4695-9bbf-c6c3d68d7329" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 14 08:01:23.403686 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:23.403657    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:23.567526 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:23.567478    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:24.258349 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:24.258315    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:24.386593 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:24.386559    2562 patch_prober.go:28] interesting pod/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 14 08:01:24.386768 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:24.386611    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt" podUID="8512cd08-a696-4695-9bbf-c6c3d68d7329" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 14 08:01:25.251418 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:25.251381    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:25.386044 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:25.386010    2562 patch_prober.go:28] interesting pod/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 14 08:01:25.386418 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:25.386069    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt" podUID="8512cd08-a696-4695-9bbf-c6c3d68d7329" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 14 08:01:26.027243 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:26.027211    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:26.386048 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:26.385956    2562 patch_prober.go:28] interesting pod/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 14 08:01:26.386048 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:26.386024    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt" podUID="8512cd08-a696-4695-9bbf-c6c3d68d7329" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 14 08:01:27.039838 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:27.039795    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:27.386560 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:27.386454    2562 patch_prober.go:28] interesting pod/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Sep 14 08:01:27.386560 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:27.386545    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt" podUID="8512cd08-a696-4695-9bbf-c6c3d68d7329" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Sep 14 08:01:27.564754 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:27.564707    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:01:27.780343 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:27.780298    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:28.390785 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:28.390752    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:01:28.391922 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:28.391902    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:01:28.598512 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:28.598467    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:29.212404 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:29.212366    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:01:33.288976 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.288934    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"]
Sep 14 08:01:33.289580 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.289555    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="110b1602-7bad-4bd0-8175-54982ec82282" containerName="istio-proxy"
Sep 14 08:01:33.289580 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.289578    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="110b1602-7bad-4bd0-8175-54982ec82282" containerName="istio-proxy"
Sep 14 08:01:33.289751 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.289596    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="110b1602-7bad-4bd0-8175-54982ec82282" containerName="istio-proxy"
Sep 14 08:01:33.289751 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.289603    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="110b1602-7bad-4bd0-8175-54982ec82282" containerName="istio-proxy"
Sep 14 08:01:33.289751 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.289674    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="110b1602-7bad-4bd0-8175-54982ec82282" containerName="istio-proxy"
Sep 14 08:01:33.292350 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.292333    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:33.294959 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.294932    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-b69a34\"/\"openshift-service-ca.crt\""
Sep 14 08:01:33.295079 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.294958    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b69a34\"/\"default-dockercfg-kq58t\""
Sep 14 08:01:33.295079 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.294991    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-b69a34\"/\"kube-root-ca.crt\""
Sep 14 08:01:33.295079 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.294945    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b69a34\"/\"simulator-w4-b69a34-kserve-self-signed-certs\""
Sep 14 08:01:33.302270 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.302248    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"]
Sep 14 08:01:33.376162 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.376107    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-dshm\") pod \"simulator-w4-b69a34-kserve-68455bf9c4-c2ktt\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") " pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:33.376162 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.376179    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd70e9fd-10f3-4b03-be62-7e130ad88022-tls-certs\") pod \"simulator-w4-b69a34-kserve-68455bf9c4-c2ktt\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") " pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:33.376391 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.376259    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d98sl\" (UniqueName: \"kubernetes.io/projected/bd70e9fd-10f3-4b03-be62-7e130ad88022-kube-api-access-d98sl\") pod \"simulator-w4-b69a34-kserve-68455bf9c4-c2ktt\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") " pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:33.376391 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.376303    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-model-cache\") pod \"simulator-w4-b69a34-kserve-68455bf9c4-c2ktt\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") " pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:33.376460 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.376379    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-home\") pod \"simulator-w4-b69a34-kserve-68455bf9c4-c2ktt\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") " pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:33.376460 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.376440    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-tmp-dir\") pod \"simulator-w4-b69a34-kserve-68455bf9c4-c2ktt\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") " pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:33.477257 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.477223    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-tmp-dir\") pod \"simulator-w4-b69a34-kserve-68455bf9c4-c2ktt\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") " pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:33.477257 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.477275    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-dshm\") pod \"simulator-w4-b69a34-kserve-68455bf9c4-c2ktt\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") " pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:33.477537 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.477314    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd70e9fd-10f3-4b03-be62-7e130ad88022-tls-certs\") pod \"simulator-w4-b69a34-kserve-68455bf9c4-c2ktt\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") " pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:33.477537 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.477350    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d98sl\" (UniqueName: \"kubernetes.io/projected/bd70e9fd-10f3-4b03-be62-7e130ad88022-kube-api-access-d98sl\") pod \"simulator-w4-b69a34-kserve-68455bf9c4-c2ktt\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") " pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:33.477537 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.477380    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-model-cache\") pod \"simulator-w4-b69a34-kserve-68455bf9c4-c2ktt\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") " pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:33.477537 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.477412    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-home\") pod \"simulator-w4-b69a34-kserve-68455bf9c4-c2ktt\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") " pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:33.477759 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.477702    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-tmp-dir\") pod \"simulator-w4-b69a34-kserve-68455bf9c4-c2ktt\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") " pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:33.477964 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.477942    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-home\") pod \"simulator-w4-b69a34-kserve-68455bf9c4-c2ktt\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") " pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:33.478040 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.477997    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-model-cache\") pod \"simulator-w4-b69a34-kserve-68455bf9c4-c2ktt\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") " pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:33.479610 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.479581    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-dshm\") pod \"simulator-w4-b69a34-kserve-68455bf9c4-c2ktt\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") " pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:33.480397 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.480360    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd70e9fd-10f3-4b03-be62-7e130ad88022-tls-certs\") pod \"simulator-w4-b69a34-kserve-68455bf9c4-c2ktt\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") " pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:33.485735 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.485714    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d98sl\" (UniqueName: \"kubernetes.io/projected/bd70e9fd-10f3-4b03-be62-7e130ad88022-kube-api-access-d98sl\") pod \"simulator-w4-b69a34-kserve-68455bf9c4-c2ktt\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") " pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:33.501268 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.501241    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:33.602356 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.602256    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:33.904436 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:33.904365    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"]
Sep 14 08:01:33.907113 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:01:33.907074    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd70e9fd_10f3_4b03_be62_7e130ad88022.slice/crio-e6ab544a1c19d873648dc07baa15419dabab0ecd281dfdfe87bb6da56882b43e WatchSource:0}: Error finding container e6ab544a1c19d873648dc07baa15419dabab0ecd281dfdfe87bb6da56882b43e: Status 404 returned error can't find the container with id e6ab544a1c19d873648dc07baa15419dabab0ecd281dfdfe87bb6da56882b43e
Sep 14 08:01:34.785967 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:34.785928    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt" event={"ID":"bd70e9fd-10f3-4b03-be62-7e130ad88022","Type":"ContainerStarted","Data":"29c18e1a93f9b8ff81ca6c24fd5f9414a8d602ba4d37a525181b13787f2f7db8"}
Sep 14 08:01:34.786441 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:34.785978    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt" event={"ID":"bd70e9fd-10f3-4b03-be62-7e130ad88022","Type":"ContainerStarted","Data":"e6ab544a1c19d873648dc07baa15419dabab0ecd281dfdfe87bb6da56882b43e"}
Sep 14 08:01:34.817035 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:34.816979    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt" podStartSLOduration=1.81696301 podStartE2EDuration="1.81696301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:01:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:01:34.81427369 +0000 UTC m=+797.270723684" watchObservedRunningTime="2026-09-14 08:01:34.81696301 +0000 UTC m=+797.273413005"
Sep 14 08:01:39.815113 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:39.815078    2562 generic.go:356] "Generic (PLEG): container finished" podID="7e2e34e5-5170-4e7c-803b-7940b1e12a09" containerID="f386eea92af836bec1ff5416df087ef07fadc1a0f03adfd731423f6b1ee0b2f7" exitCode=0
Sep 14 08:01:39.815559 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:39.815145    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx" event={"ID":"7e2e34e5-5170-4e7c-803b-7940b1e12a09","Type":"ContainerDied","Data":"f386eea92af836bec1ff5416df087ef07fadc1a0f03adfd731423f6b1ee0b2f7"}
Sep 14 08:01:39.815610 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:39.815585    2562 scope.go:117] "RemoveContainer" containerID="f386eea92af836bec1ff5416df087ef07fadc1a0f03adfd731423f6b1ee0b2f7"
Sep 14 08:01:39.816870 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:39.816842    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:01:39.816935 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:39.816913    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:01:39.816971 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:39.816947    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:01:40.822769 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:40.822675    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx" event={"ID":"7e2e34e5-5170-4e7c-803b-7940b1e12a09","Type":"ContainerStarted","Data":"6767806daac5acc3451e65bb7b8bc61ef579d6785a9e0a0dcde66d90e508c41e"}
Sep 14 08:01:41.702009 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:41.701962    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:41.702198 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:41.702160    2562 patch_prober.go:28] interesting pod/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 14 08:01:41.702245 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:41.702217    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx" podUID="7e2e34e5-5170-4e7c-803b-7940b1e12a09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 14 08:01:42.522139 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:42.522105    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:42.702407 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:42.702373    2562 patch_prober.go:28] interesting pod/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 14 08:01:42.702599 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:42.702438    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx" podUID="7e2e34e5-5170-4e7c-803b-7940b1e12a09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 14 08:01:43.602614 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:43.602574    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:43.603004 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:43.602632    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:43.615465 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:43.615431    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:43.702143 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:43.702103    2562 patch_prober.go:28] interesting pod/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 14 08:01:43.702320 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:43.702162    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx" podUID="7e2e34e5-5170-4e7c-803b-7940b1e12a09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 14 08:01:43.873095 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:43.872995    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:01:44.188741 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:44.188702    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-71665efb-x644m"]
Sep 14 08:01:44.189212 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:44.189187    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="110b1602-7bad-4bd0-8175-54982ec82282" containerName="istio-proxy"
Sep 14 08:01:44.191135 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:44.191115    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-71665efb-x644m"
Sep 14 08:01:44.193361 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:44.193339    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:01:44.193361 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:44.193343    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:01:44.199802 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:44.199780    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-71665efb-x644m"]
Sep 14 08:01:44.290410 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:44.290371    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/402e7ef4-db0a-412c-8edd-ee5a95f6e545-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-71665efb-x644m\" (UID: \"402e7ef4-db0a-412c-8edd-ee5a95f6e545\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-71665efb-x644m"
Sep 14 08:01:44.391139 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:44.391080    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/402e7ef4-db0a-412c-8edd-ee5a95f6e545-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-71665efb-x644m\" (UID: \"402e7ef4-db0a-412c-8edd-ee5a95f6e545\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-71665efb-x644m"
Sep 14 08:01:44.391797 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:44.391777    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/402e7ef4-db0a-412c-8edd-ee5a95f6e545-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-71665efb-x644m\" (UID: \"402e7ef4-db0a-412c-8edd-ee5a95f6e545\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-71665efb-x644m"
Sep 14 08:01:44.503004 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:44.502912    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-71665efb-x644m"
Sep 14 08:01:44.702095 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:44.702051    2562 patch_prober.go:28] interesting pod/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 14 08:01:44.702490 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:44.702124    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx" podUID="7e2e34e5-5170-4e7c-803b-7940b1e12a09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 14 08:01:44.797163 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:44.797135    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-71665efb-x644m"]
Sep 14 08:01:44.799406 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:01:44.799378    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod402e7ef4_db0a_412c_8edd_ee5a95f6e545.slice/crio-799884c58595a32cf4a26eee26424f733dca867f34973f5d10d08c56824f5b05 WatchSource:0}: Error finding container 799884c58595a32cf4a26eee26424f733dca867f34973f5d10d08c56824f5b05: Status 404 returned error can't find the container with id 799884c58595a32cf4a26eee26424f733dca867f34973f5d10d08c56824f5b05
Sep 14 08:01:44.847447 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:44.847402    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-71665efb-x644m" event={"ID":"402e7ef4-db0a-412c-8edd-ee5a95f6e545","Type":"ContainerStarted","Data":"799884c58595a32cf4a26eee26424f733dca867f34973f5d10d08c56824f5b05"}
Sep 14 08:01:45.256342 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:45.256306    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:45.317528 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:45.317471    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:45.702349 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:45.702308    2562 patch_prober.go:28] interesting pod/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Sep 14 08:01:45.702745 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:45.702368    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx" podUID="7e2e34e5-5170-4e7c-803b-7940b1e12a09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Sep 14 08:01:46.013808 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:46.013711    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:46.706544 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:46.706511    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:46.706921 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:46.706798    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:46.707588 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:46.707567    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:01:47.247335 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:47.247283    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:47.809418 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:47.809378    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:01:47.834917 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:47.834871    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:48.758177 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:48.758140    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:48.873600 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:48.873567    2562 generic.go:356] "Generic (PLEG): container finished" podID="402e7ef4-db0a-412c-8edd-ee5a95f6e545" containerID="35b292d156d9a78416af31c91cc99490c87c23f0cc729d49a1ea53a79e3e06ce" exitCode=0
Sep 14 08:01:48.874021 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:48.873611    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-71665efb-x644m" event={"ID":"402e7ef4-db0a-412c-8edd-ee5a95f6e545","Type":"ContainerDied","Data":"35b292d156d9a78416af31c91cc99490c87c23f0cc729d49a1ea53a79e3e06ce"}
Sep 14 08:01:49.847230 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:49.847195    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:01:50.019006 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:50.018981    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-71665efb-x644m"
Sep 14 08:01:50.048575 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:50.048545    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/402e7ef4-db0a-412c-8edd-ee5a95f6e545-maas-api-service-ca\") pod \"402e7ef4-db0a-412c-8edd-ee5a95f6e545\" (UID: \"402e7ef4-db0a-412c-8edd-ee5a95f6e545\") "
Sep 14 08:01:50.048875 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:50.048853    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/402e7ef4-db0a-412c-8edd-ee5a95f6e545-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "402e7ef4-db0a-412c-8edd-ee5a95f6e545" (UID: "402e7ef4-db0a-412c-8edd-ee5a95f6e545"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:01:50.150310 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:50.150274    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/402e7ef4-db0a-412c-8edd-ee5a95f6e545-maas-api-service-ca\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:01:50.889896 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:50.889850    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-71665efb-x644m"
Sep 14 08:01:50.889896 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:50.889889    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-71665efb-x644m" event={"ID":"402e7ef4-db0a-412c-8edd-ee5a95f6e545","Type":"ContainerDied","Data":"799884c58595a32cf4a26eee26424f733dca867f34973f5d10d08c56824f5b05"}
Sep 14 08:01:50.890113 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:50.889967    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="799884c58595a32cf4a26eee26424f733dca867f34973f5d10d08c56824f5b05"
Sep 14 08:01:51.178092 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:01:51.178003    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:00.423770 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:00.423728    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:01.132337 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:01.132300    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:01.233406 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:01.233371    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:01.333397 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:01.333356    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:01.357271 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:01.357240    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-71665efb-x644m"]
Sep 14 08:02:01.359558 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:01.359536    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-71665efb-x644m"]
Sep 14 08:02:01.459467 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:01.459421    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:01.497572 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:01.497542    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"]
Sep 14 08:02:01.497821 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:01.497793    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx" podUID="7e2e34e5-5170-4e7c-803b-7940b1e12a09" containerName="istio-proxy" containerID="cri-o://6767806daac5acc3451e65bb7b8bc61ef579d6785a9e0a0dcde66d90e508c41e" gracePeriod=30
Sep 14 08:02:01.552914 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:01.552882    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:01.709670 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:01.709583    2562 patch_prober.go:28] interesting pod/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 08:02:01.709811 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:01.709663    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx" podUID="7e2e34e5-5170-4e7c-803b-7940b1e12a09" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 08:02:02.113222 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.113144    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="402e7ef4-db0a-412c-8edd-ee5a95f6e545" path="/var/lib/kubelet/pods/402e7ef4-db0a-412c-8edd-ee5a95f6e545/volumes"
Sep 14 08:02:02.274018 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.273986    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"]
Sep 14 08:02:02.274485 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.274466    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="402e7ef4-db0a-412c-8edd-ee5a95f6e545" containerName="revoke-keys"
Sep 14 08:02:02.274609 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.274488    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="402e7ef4-db0a-412c-8edd-ee5a95f6e545" containerName="revoke-keys"
Sep 14 08:02:02.274671 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.274622    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="402e7ef4-db0a-412c-8edd-ee5a95f6e545" containerName="revoke-keys"
Sep 14 08:02:02.279999 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.279976    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.282218 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.282190    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-fd181c19-openshift-default-dockercfg-sk8dk\""
Sep 14 08:02:02.289651 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.289622    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"]
Sep 14 08:02:02.373580 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.369629    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-envoy\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.373580 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.369781    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-data\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.373580 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.369840    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4c9b\" (UniqueName: \"kubernetes.io/projected/b6a5522b-42fe-4030-bf46-852fc4382fe6-kube-api-access-b4c9b\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.373580 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.369934    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-credential-socket\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.373580 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.369995    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-workload-socket\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.373580 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.370118    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6a5522b-42fe-4030-bf46-852fc4382fe6-istiod-ca-cert\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.373580 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.370209    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-podinfo\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.373580 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.370277    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-workload-certs\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.373580 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.370300    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-token\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.471631 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.471587    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-workload-certs\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.471631 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.471631    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-token\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.472159 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.471654    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-envoy\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.472159 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.471707    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-data\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.472159 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.471732    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4c9b\" (UniqueName: \"kubernetes.io/projected/b6a5522b-42fe-4030-bf46-852fc4382fe6-kube-api-access-b4c9b\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.472159 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.471785    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-credential-socket\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.472159 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.471815    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-workload-socket\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.472159 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.471881    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6a5522b-42fe-4030-bf46-852fc4382fe6-istiod-ca-cert\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.472159 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.471943    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-podinfo\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.472159 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.472058    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-workload-certs\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.472625 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.472396    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-credential-socket\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.472625 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.472449    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-workload-socket\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.472625 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.472486    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-data\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.472780 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.472723    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6a5522b-42fe-4030-bf46-852fc4382fe6-istiod-ca-cert\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.474066 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.474046    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-envoy\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.474327 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.474308    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-podinfo\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.481407 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.481372    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-token\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.482485 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.482458    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4c9b\" (UniqueName: \"kubernetes.io/projected/b6a5522b-42fe-4030-bf46-852fc4382fe6-kube-api-access-b4c9b\") pod \"e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") " pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.503470 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.503427    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:02.592765 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.592728    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:02.604791 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.604764    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:02.703434 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.703400    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:02.809939 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.809905    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:02.882073 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.882040    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"]
Sep 14 08:02:02.885848 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:02:02.885820    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6a5522b_42fe_4030_bf46_852fc4382fe6.slice/crio-dcd9332580bdf84b70c550c0f29e3b4551f2c351eb5d3d756095019327768352 WatchSource:0}: Error finding container dcd9332580bdf84b70c550c0f29e3b4551f2c351eb5d3d756095019327768352: Status 404 returned error can't find the container with id dcd9332580bdf84b70c550c0f29e3b4551f2c351eb5d3d756095019327768352
Sep 14 08:02:02.888309 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.888276    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:02:02.888394 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.888342    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:02:02.888394 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.888371    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:02:02.951638 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:02.951613    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr" event={"ID":"b6a5522b-42fe-4030-bf46-852fc4382fe6","Type":"ContainerStarted","Data":"dcd9332580bdf84b70c550c0f29e3b4551f2c351eb5d3d756095019327768352"}
Sep 14 08:02:03.755244 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:03.755204    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:03.962035 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:03.961999    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr" event={"ID":"b6a5522b-42fe-4030-bf46-852fc4382fe6","Type":"ContainerStarted","Data":"4dce9a51d47869d58984f990f88a0df657dfa02be3c81128ebb7c15c71e5c22e"}
Sep 14 08:02:03.986831 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:03.986781    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr" podStartSLOduration=1.986763706 podStartE2EDuration="1.986763706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:03.984754643 +0000 UTC m=+826.441204651" watchObservedRunningTime="2026-09-14 08:02:03.986763706 +0000 UTC m=+826.443213770"
Sep 14 08:02:04.593069 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:04.593032    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:04.598173 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:04.598146    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:04.732855 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:04.732806    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:04.967571 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:04.967533    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:04.968525 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:04.968491    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:05.335596 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:05.335492    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:05.470423 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:05.470385    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:05.666554 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:05.666440    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:06.756986 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.756959    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:02:06.818269 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.818186    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:06.818269 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.818257    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istiod-ca-cert\") pod \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") "
Sep 14 08:02:06.818492 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.818312    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-envoy\") pod \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") "
Sep 14 08:02:06.818492 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.818343    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9hcn\" (UniqueName: \"kubernetes.io/projected/7e2e34e5-5170-4e7c-803b-7940b1e12a09-kube-api-access-b9hcn\") pod \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") "
Sep 14 08:02:06.818492 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.818369    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-podinfo\") pod \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") "
Sep 14 08:02:06.818492 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.818433    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-token\") pod \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") "
Sep 14 08:02:06.818492 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.818480    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-workload-socket\") pod \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") "
Sep 14 08:02:06.818781 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.818556    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-credential-socket\") pod \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") "
Sep 14 08:02:06.818781 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.818584    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-data\") pod \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") "
Sep 14 08:02:06.818781 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.818625    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-workload-certs\") pod \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\" (UID: \"7e2e34e5-5170-4e7c-803b-7940b1e12a09\") "
Sep 14 08:02:06.818781 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.818653    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7e2e34e5-5170-4e7c-803b-7940b1e12a09" (UID: "7e2e34e5-5170-4e7c-803b-7940b1e12a09"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:02:06.818982 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.818935    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7e2e34e5-5170-4e7c-803b-7940b1e12a09" (UID: "7e2e34e5-5170-4e7c-803b-7940b1e12a09"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:02:06.819071 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.819045    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-data" (OuterVolumeSpecName: "istio-data") pod "7e2e34e5-5170-4e7c-803b-7940b1e12a09" (UID: "7e2e34e5-5170-4e7c-803b-7940b1e12a09"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:02:06.819135 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.819062    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-credential-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:02:06.819135 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.819124    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istiod-ca-cert\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:02:06.819229 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.819128    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7e2e34e5-5170-4e7c-803b-7940b1e12a09" (UID: "7e2e34e5-5170-4e7c-803b-7940b1e12a09"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:02:06.819423 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.819397    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7e2e34e5-5170-4e7c-803b-7940b1e12a09" (UID: "7e2e34e5-5170-4e7c-803b-7940b1e12a09"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:02:06.820776 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.820749    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e2e34e5-5170-4e7c-803b-7940b1e12a09-kube-api-access-b9hcn" (OuterVolumeSpecName: "kube-api-access-b9hcn") pod "7e2e34e5-5170-4e7c-803b-7940b1e12a09" (UID: "7e2e34e5-5170-4e7c-803b-7940b1e12a09"). InnerVolumeSpecName "kube-api-access-b9hcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:02:06.820929 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.820906    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-token" (OuterVolumeSpecName: "istio-token") pod "7e2e34e5-5170-4e7c-803b-7940b1e12a09" (UID: "7e2e34e5-5170-4e7c-803b-7940b1e12a09"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:02:06.821046 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.821027    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7e2e34e5-5170-4e7c-803b-7940b1e12a09" (UID: "7e2e34e5-5170-4e7c-803b-7940b1e12a09"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:02:06.821105 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.821083    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7e2e34e5-5170-4e7c-803b-7940b1e12a09" (UID: "7e2e34e5-5170-4e7c-803b-7940b1e12a09"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:02:06.920017 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.919984    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-token\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:02:06.920017 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.920017    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-workload-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:02:06.920220 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.920027    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-data\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:02:06.920220 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.920036    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-workload-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:02:06.920220 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.920044    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-envoy\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:02:06.920220 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.920052    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9hcn\" (UniqueName: \"kubernetes.io/projected/7e2e34e5-5170-4e7c-803b-7940b1e12a09-kube-api-access-b9hcn\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:02:06.920220 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.920060    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e2e34e5-5170-4e7c-803b-7940b1e12a09-istio-podinfo\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:02:06.978806 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.978772    2562 generic.go:356] "Generic (PLEG): container finished" podID="7e2e34e5-5170-4e7c-803b-7940b1e12a09" containerID="6767806daac5acc3451e65bb7b8bc61ef579d6785a9e0a0dcde66d90e508c41e" exitCode=0
Sep 14 08:02:06.978990 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.978827    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"
Sep 14 08:02:06.978990 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.978848    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx" event={"ID":"7e2e34e5-5170-4e7c-803b-7940b1e12a09","Type":"ContainerDied","Data":"6767806daac5acc3451e65bb7b8bc61ef579d6785a9e0a0dcde66d90e508c41e"}
Sep 14 08:02:06.978990 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.978912    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx" event={"ID":"7e2e34e5-5170-4e7c-803b-7940b1e12a09","Type":"ContainerDied","Data":"7a89ec100d2e20d876e546781fb01a1df2da873151413ad2f93b1409e34c5265"}
Sep 14 08:02:06.978990 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.978945    2562 scope.go:117] "RemoveContainer" containerID="6767806daac5acc3451e65bb7b8bc61ef579d6785a9e0a0dcde66d90e508c41e"
Sep 14 08:02:06.988520 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.988485    2562 scope.go:117] "RemoveContainer" containerID="f386eea92af836bec1ff5416df087ef07fadc1a0f03adfd731423f6b1ee0b2f7"
Sep 14 08:02:06.996521 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.996489    2562 scope.go:117] "RemoveContainer" containerID="6767806daac5acc3451e65bb7b8bc61ef579d6785a9e0a0dcde66d90e508c41e"
Sep 14 08:02:06.996823 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:02:06.996803    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6767806daac5acc3451e65bb7b8bc61ef579d6785a9e0a0dcde66d90e508c41e\": container with ID starting with 6767806daac5acc3451e65bb7b8bc61ef579d6785a9e0a0dcde66d90e508c41e not found: ID does not exist" containerID="6767806daac5acc3451e65bb7b8bc61ef579d6785a9e0a0dcde66d90e508c41e"
Sep 14 08:02:06.996872 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.996833    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6767806daac5acc3451e65bb7b8bc61ef579d6785a9e0a0dcde66d90e508c41e"} err="failed to get container status \"6767806daac5acc3451e65bb7b8bc61ef579d6785a9e0a0dcde66d90e508c41e\": rpc error: code = NotFound desc = could not find container \"6767806daac5acc3451e65bb7b8bc61ef579d6785a9e0a0dcde66d90e508c41e\": container with ID starting with 6767806daac5acc3451e65bb7b8bc61ef579d6785a9e0a0dcde66d90e508c41e not found: ID does not exist"
Sep 14 08:02:06.996872 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.996849    2562 scope.go:117] "RemoveContainer" containerID="f386eea92af836bec1ff5416df087ef07fadc1a0f03adfd731423f6b1ee0b2f7"
Sep 14 08:02:06.997104 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:02:06.997084    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f386eea92af836bec1ff5416df087ef07fadc1a0f03adfd731423f6b1ee0b2f7\": container with ID starting with f386eea92af836bec1ff5416df087ef07fadc1a0f03adfd731423f6b1ee0b2f7 not found: ID does not exist" containerID="f386eea92af836bec1ff5416df087ef07fadc1a0f03adfd731423f6b1ee0b2f7"
Sep 14 08:02:06.997155 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:06.997107    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f386eea92af836bec1ff5416df087ef07fadc1a0f03adfd731423f6b1ee0b2f7"} err="failed to get container status \"f386eea92af836bec1ff5416df087ef07fadc1a0f03adfd731423f6b1ee0b2f7\": rpc error: code = NotFound desc = could not find container \"f386eea92af836bec1ff5416df087ef07fadc1a0f03adfd731423f6b1ee0b2f7\": container with ID starting with f386eea92af836bec1ff5416df087ef07fadc1a0f03adfd731423f6b1ee0b2f7 not found: ID does not exist"
Sep 14 08:02:07.005151 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:07.005125    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"]
Sep 14 08:02:07.010565 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:07.010544    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-71665efb-gw-openshift-default-776f67677b-c42mx"]
Sep 14 08:02:07.854089 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:07.854042    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:07.959192 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:07.959153    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:08.085103 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:08.085064    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:08.113136 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:08.113055    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e2e34e5-5170-4e7c-803b-7940b1e12a09" path="/var/lib/kubelet/pods/7e2e34e5-5170-4e7c-803b-7940b1e12a09/volumes"
Sep 14 08:02:08.322753 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:08.322711    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:02:08.408735 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:08.408651    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:08.507410 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:08.507372    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:09.291526 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:09.291465    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:09.929876 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:09.929838    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:10.888441 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:10.888405    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:11.701703 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:11.701665    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:23.615410 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:23.615362    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:02:28.101250 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:28.101212    2562 generic.go:356] "Generic (PLEG): container finished" podID="b6a5522b-42fe-4030-bf46-852fc4382fe6" containerID="4dce9a51d47869d58984f990f88a0df657dfa02be3c81128ebb7c15c71e5c22e" exitCode=0
Sep 14 08:02:28.101807 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:28.101305    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr" event={"ID":"b6a5522b-42fe-4030-bf46-852fc4382fe6","Type":"ContainerDied","Data":"4dce9a51d47869d58984f990f88a0df657dfa02be3c81128ebb7c15c71e5c22e"}
Sep 14 08:02:28.101903 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:28.101885    2562 scope.go:117] "RemoveContainer" containerID="4dce9a51d47869d58984f990f88a0df657dfa02be3c81128ebb7c15c71e5c22e"
Sep 14 08:02:28.103127 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:28.103097    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:02:28.103207 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:28.103165    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:02:28.103207 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:28.103197    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:02:29.108205 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:29.108171    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr" event={"ID":"b6a5522b-42fe-4030-bf46-852fc4382fe6","Type":"ContainerStarted","Data":"a3c1d0e9b3d93fc49b2daeeff5e6cd0d195d236545c2a1bf160403b815d93f20"}
Sep 14 08:02:29.593346 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:29.593301    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:29.593635 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:29.593612    2562 patch_prober.go:28] interesting pod/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 14 08:02:29.593692 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:29.593660    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr" podUID="b6a5522b-42fe-4030-bf46-852fc4382fe6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 14 08:02:30.095163 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:30.095124    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:30.593314 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:30.593272    2562 patch_prober.go:28] interesting pod/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 14 08:02:30.593738 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:30.593336    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr" podUID="b6a5522b-42fe-4030-bf46-852fc4382fe6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 14 08:02:30.881729 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:30.881643    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:31.205339 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:31.205308    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd181c19-665rz"]
Sep 14 08:02:31.205797 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:31.205780    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e2e34e5-5170-4e7c-803b-7940b1e12a09" containerName="istio-proxy"
Sep 14 08:02:31.205797 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:31.205798    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e2e34e5-5170-4e7c-803b-7940b1e12a09" containerName="istio-proxy"
Sep 14 08:02:31.205942 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:31.205818    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e2e34e5-5170-4e7c-803b-7940b1e12a09" containerName="istio-proxy"
Sep 14 08:02:31.205942 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:31.205824    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e2e34e5-5170-4e7c-803b-7940b1e12a09" containerName="istio-proxy"
Sep 14 08:02:31.205942 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:31.205885    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e2e34e5-5170-4e7c-803b-7940b1e12a09" containerName="istio-proxy"
Sep 14 08:02:31.207793 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:31.207778    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd181c19-665rz"
Sep 14 08:02:31.213474 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:31.213444    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:02:31.214414 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:31.214393    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:02:31.231837 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:31.231802    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd181c19-665rz"]
Sep 14 08:02:31.273188 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:31.273142    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e87b132-32a2-4dad-abe4-2e5b8df29f07-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fd181c19-665rz\" (UID: \"8e87b132-32a2-4dad-abe4-2e5b8df29f07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd181c19-665rz"
Sep 14 08:02:31.373967 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:31.373926    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e87b132-32a2-4dad-abe4-2e5b8df29f07-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fd181c19-665rz\" (UID: \"8e87b132-32a2-4dad-abe4-2e5b8df29f07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd181c19-665rz"
Sep 14 08:02:31.374764 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:31.374737    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e87b132-32a2-4dad-abe4-2e5b8df29f07-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fd181c19-665rz\" (UID: \"8e87b132-32a2-4dad-abe4-2e5b8df29f07\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd181c19-665rz"
Sep 14 08:02:31.527395 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:31.527296    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd181c19-665rz"
Sep 14 08:02:31.593679 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:31.593637    2562 patch_prober.go:28] interesting pod/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 14 08:02:31.594092 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:31.593707    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr" podUID="b6a5522b-42fe-4030-bf46-852fc4382fe6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 14 08:02:31.817770 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:31.817736    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd181c19-665rz"]
Sep 14 08:02:31.820521 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:02:31.820481    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e87b132_32a2_4dad_abe4_2e5b8df29f07.slice/crio-9d0f1281c82b501c784cb6a613eab064bb0c3eea350dd963d392b3c382847bdf WatchSource:0}: Error finding container 9d0f1281c82b501c784cb6a613eab064bb0c3eea350dd963d392b3c382847bdf: Status 404 returned error can't find the container with id 9d0f1281c82b501c784cb6a613eab064bb0c3eea350dd963d392b3c382847bdf
Sep 14 08:02:31.864713 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:31.864673    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:32.125065 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:32.125031    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd181c19-665rz" event={"ID":"8e87b132-32a2-4dad-abe4-2e5b8df29f07","Type":"ContainerStarted","Data":"9d0f1281c82b501c784cb6a613eab064bb0c3eea350dd963d392b3c382847bdf"}
Sep 14 08:02:32.593296 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:32.593256    2562 patch_prober.go:28] interesting pod/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 14 08:02:32.593489 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:32.593315    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr" podUID="b6a5522b-42fe-4030-bf46-852fc4382fe6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 14 08:02:32.680124 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:32.680085    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:33.132040 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:33.131949    2562 generic.go:356] "Generic (PLEG): container finished" podID="8e87b132-32a2-4dad-abe4-2e5b8df29f07" containerID="e026fe8ccc6880541df040e4916349665d21f45cd2e3f2be4b6c99cf5538a7e5" exitCode=0
Sep 14 08:02:33.132183 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:33.132037    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd181c19-665rz" event={"ID":"8e87b132-32a2-4dad-abe4-2e5b8df29f07","Type":"ContainerDied","Data":"e026fe8ccc6880541df040e4916349665d21f45cd2e3f2be4b6c99cf5538a7e5"}
Sep 14 08:02:33.594125 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:33.594080    2562 patch_prober.go:28] interesting pod/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 14 08:02:33.594311 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:33.594148    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr" podUID="b6a5522b-42fe-4030-bf46-852fc4382fe6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 14 08:02:33.650399 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:33.650365    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:34.320043 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:34.320018    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd181c19-665rz"
Sep 14 08:02:34.404728 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:34.404618    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e87b132-32a2-4dad-abe4-2e5b8df29f07-maas-api-service-ca\") pod \"8e87b132-32a2-4dad-abe4-2e5b8df29f07\" (UID: \"8e87b132-32a2-4dad-abe4-2e5b8df29f07\") "
Sep 14 08:02:34.404991 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:34.404963    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e87b132-32a2-4dad-abe4-2e5b8df29f07-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8e87b132-32a2-4dad-abe4-2e5b8df29f07" (UID: "8e87b132-32a2-4dad-abe4-2e5b8df29f07"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:02:34.506127 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:34.506086    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8e87b132-32a2-4dad-abe4-2e5b8df29f07-maas-api-service-ca\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:02:34.564825 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:34.564789    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:34.670529 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:34.670420    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:34.670783 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:34.670763    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:34.671850 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:34.671824    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:35.039055 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:35.039022    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:35.144226 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:35.144195    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd181c19-665rz"
Sep 14 08:02:35.144409 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:35.144238    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd181c19-665rz" event={"ID":"8e87b132-32a2-4dad-abe4-2e5b8df29f07","Type":"ContainerDied","Data":"9d0f1281c82b501c784cb6a613eab064bb0c3eea350dd963d392b3c382847bdf"}
Sep 14 08:02:35.144409 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:35.144291    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d0f1281c82b501c784cb6a613eab064bb0c3eea350dd963d392b3c382847bdf"
Sep 14 08:02:35.417524 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:35.417410    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:36.650943 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:36.650910    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:37.456075 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:37.456038    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:38.282603 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:38.282563    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:38.386123 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:38.386092    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:38.469615 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:38.469540    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd181c19-665rz"]
Sep 14 08:02:38.472150 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:38.472119    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:38.472398 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:38.472379    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fd181c19-665rz"]
Sep 14 08:02:38.529705 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:38.529671    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:38.944617 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:38.944582    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:39.625214 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:39.625179    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"]
Sep 14 08:02:39.625638 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:39.625418    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr" podUID="b6a5522b-42fe-4030-bf46-852fc4382fe6" containerName="istio-proxy" containerID="cri-o://a3c1d0e9b3d93fc49b2daeeff5e6cd0d195d236545c2a1bf160403b815d93f20" gracePeriod=30
Sep 14 08:02:39.646571 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:39.646542    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:39.834549 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:39.834495    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:40.091029 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.090989    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:40.113010 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.112979    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e87b132-32a2-4dad-abe4-2e5b8df29f07" path="/var/lib/kubelet/pods/8e87b132-32a2-4dad-abe4-2e5b8df29f07/volumes"
Sep 14 08:02:40.376561 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.376449    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"]
Sep 14 08:02:40.376893 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.376877    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e87b132-32a2-4dad-abe4-2e5b8df29f07" containerName="revoke-keys"
Sep 14 08:02:40.376962 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.376894    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e87b132-32a2-4dad-abe4-2e5b8df29f07" containerName="revoke-keys"
Sep 14 08:02:40.377023 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.376963    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e87b132-32a2-4dad-abe4-2e5b8df29f07" containerName="revoke-keys"
Sep 14 08:02:40.377023 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.376980    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e2e34e5-5170-4e7c-803b-7940b1e12a09" containerName="istio-proxy"
Sep 14 08:02:40.379173 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.379157    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.381586 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.381565    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-b931e88a-openshift-default-dockercfg-cj7jb\""
Sep 14 08:02:40.391676 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.391651    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"]
Sep 14 08:02:40.397476 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.397446    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:40.468357 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.468308    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82w9h\" (UniqueName: \"kubernetes.io/projected/75b332e7-a761-4796-8d34-3e2f4382c41e-kube-api-access-82w9h\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.468662 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.468628    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-envoy\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.468893 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.468875    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-workload-socket\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.469081 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.469052    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-workload-certs\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.469219 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.469204    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75b332e7-a761-4796-8d34-3e2f4382c41e-istiod-ca-cert\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.469465 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.469435    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-credential-socket\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.469722 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.469701    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-podinfo\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.469901 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.469876    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-token\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.470417 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.470398    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-data\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.572139 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.572098    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-workload-certs\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.572139 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.572146    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75b332e7-a761-4796-8d34-3e2f4382c41e-istiod-ca-cert\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.572387 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.572172    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-credential-socket\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.572387 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.572190    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-podinfo\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.572387 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.572222    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-token\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.572387 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.572274    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-data\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.572643 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.572427    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82w9h\" (UniqueName: \"kubernetes.io/projected/75b332e7-a761-4796-8d34-3e2f4382c41e-kube-api-access-82w9h\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.572643 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.572480    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-envoy\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.572643 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.572607    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-workload-socket\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.572643 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.572624    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-credential-socket\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.572852 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.572820    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-workload-certs\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.572957 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.572936    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-data\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.573018 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.572985    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75b332e7-a761-4796-8d34-3e2f4382c41e-istiod-ca-cert\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.573082 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.573059    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-workload-socket\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.574710 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.574680    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-envoy\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.574908 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.574893    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-podinfo\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.582596 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.582573    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-token\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.586822 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.586799    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82w9h\" (UniqueName: \"kubernetes.io/projected/75b332e7-a761-4796-8d34-3e2f4382c41e-kube-api-access-82w9h\") pod \"e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") " pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.691068 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.691032    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:40.717466 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.717428    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:40.824060 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.823958    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:40.971234 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.971144    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"]
Sep 14 08:02:40.974162 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:02:40.974135    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75b332e7_a761_4796_8d34_3e2f4382c41e.slice/crio-a88ef4d89da23b6714b6a5bc7b978c26bf29aa8eadf0a2f07cabcc72fa0a2260 WatchSource:0}: Error finding container a88ef4d89da23b6714b6a5bc7b978c26bf29aa8eadf0a2f07cabcc72fa0a2260: Status 404 returned error can't find the container with id a88ef4d89da23b6714b6a5bc7b978c26bf29aa8eadf0a2f07cabcc72fa0a2260
Sep 14 08:02:40.976341 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.976312    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:02:40.976425 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.976376    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:02:40.976425 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:40.976401    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:02:41.176224 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:41.176182    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b" event={"ID":"75b332e7-a761-4796-8d34-3e2f4382c41e","Type":"ContainerStarted","Data":"c7cfd43b3b4a8fb0b59a060789602b6dd2884c56dd57fbca437f6821565cceb0"}
Sep 14 08:02:41.176426 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:41.176251    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b" event={"ID":"75b332e7-a761-4796-8d34-3e2f4382c41e","Type":"ContainerStarted","Data":"a88ef4d89da23b6714b6a5bc7b978c26bf29aa8eadf0a2f07cabcc72fa0a2260"}
Sep 14 08:02:41.201441 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:41.201390    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b" podStartSLOduration=1.201370336 podStartE2EDuration="1.201370336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:41.199056548 +0000 UTC m=+863.655506553" watchObservedRunningTime="2026-09-14 08:02:41.201370336 +0000 UTC m=+863.657820329"
Sep 14 08:02:41.543260 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:41.543227    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:41.692008 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:41.691970    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:42.211120 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:42.211076    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:02:42.324235 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:42.324200    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:42.538869 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:42.538789    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:42.696998 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:42.696968    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:43.187238 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:43.187208    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:43.188288 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:43.188267    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:02:43.639117 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:43.639037    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:43.739865 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:43.739828    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:44.383294 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:44.383260    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:44.485539 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:44.485486    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:44.584086 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:44.584049    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:44.882537 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:44.882492    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:44.908771 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:44.908650    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:45.004407 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.004372    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:45.018463 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.018432    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-token\") pod \"b6a5522b-42fe-4030-bf46-852fc4382fe6\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") "
Sep 14 08:02:45.018656 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.018497    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6a5522b-42fe-4030-bf46-852fc4382fe6-istiod-ca-cert\") pod \"b6a5522b-42fe-4030-bf46-852fc4382fe6\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") "
Sep 14 08:02:45.018656 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.018575    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-podinfo\") pod \"b6a5522b-42fe-4030-bf46-852fc4382fe6\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") "
Sep 14 08:02:45.018656 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.018603    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-workload-socket\") pod \"b6a5522b-42fe-4030-bf46-852fc4382fe6\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") "
Sep 14 08:02:45.018656 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.018632    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-credential-socket\") pod \"b6a5522b-42fe-4030-bf46-852fc4382fe6\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") "
Sep 14 08:02:45.018656 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.018656    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-workload-certs\") pod \"b6a5522b-42fe-4030-bf46-852fc4382fe6\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") "
Sep 14 08:02:45.018923 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.018686    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-envoy\") pod \"b6a5522b-42fe-4030-bf46-852fc4382fe6\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") "
Sep 14 08:02:45.018923 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.018766    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4c9b\" (UniqueName: \"kubernetes.io/projected/b6a5522b-42fe-4030-bf46-852fc4382fe6-kube-api-access-b4c9b\") pod \"b6a5522b-42fe-4030-bf46-852fc4382fe6\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") "
Sep 14 08:02:45.018923 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.018794    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-data\") pod \"b6a5522b-42fe-4030-bf46-852fc4382fe6\" (UID: \"b6a5522b-42fe-4030-bf46-852fc4382fe6\") "
Sep 14 08:02:45.019073 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.018982    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b6a5522b-42fe-4030-bf46-852fc4382fe6" (UID: "b6a5522b-42fe-4030-bf46-852fc4382fe6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:02:45.019073 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.018994    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b6a5522b-42fe-4030-bf46-852fc4382fe6" (UID: "b6a5522b-42fe-4030-bf46-852fc4382fe6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:02:45.019073 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.019009    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b6a5522b-42fe-4030-bf46-852fc4382fe6" (UID: "b6a5522b-42fe-4030-bf46-852fc4382fe6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:02:45.019222 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.019065    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6a5522b-42fe-4030-bf46-852fc4382fe6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b6a5522b-42fe-4030-bf46-852fc4382fe6" (UID: "b6a5522b-42fe-4030-bf46-852fc4382fe6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:02:45.019222 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.019203    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-data" (OuterVolumeSpecName: "istio-data") pod "b6a5522b-42fe-4030-bf46-852fc4382fe6" (UID: "b6a5522b-42fe-4030-bf46-852fc4382fe6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:02:45.019306 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.019295    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-data\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:02:45.019358 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.019313    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6a5522b-42fe-4030-bf46-852fc4382fe6-istiod-ca-cert\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:02:45.019358 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.019326    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-workload-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:02:45.019358 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.019338    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-credential-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:02:45.019358 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.019351    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-workload-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:02:45.020902 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.020874    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b6a5522b-42fe-4030-bf46-852fc4382fe6" (UID: "b6a5522b-42fe-4030-bf46-852fc4382fe6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:02:45.021068 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.021050    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6a5522b-42fe-4030-bf46-852fc4382fe6-kube-api-access-b4c9b" (OuterVolumeSpecName: "kube-api-access-b4c9b") pod "b6a5522b-42fe-4030-bf46-852fc4382fe6" (UID: "b6a5522b-42fe-4030-bf46-852fc4382fe6"). InnerVolumeSpecName "kube-api-access-b4c9b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:02:45.021193 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.021179    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b6a5522b-42fe-4030-bf46-852fc4382fe6" (UID: "b6a5522b-42fe-4030-bf46-852fc4382fe6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:02:45.021251 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.021211    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-token" (OuterVolumeSpecName: "istio-token") pod "b6a5522b-42fe-4030-bf46-852fc4382fe6" (UID: "b6a5522b-42fe-4030-bf46-852fc4382fe6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:02:45.120675 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.120640    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-envoy\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:02:45.120675 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.120673    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4c9b\" (UniqueName: \"kubernetes.io/projected/b6a5522b-42fe-4030-bf46-852fc4382fe6-kube-api-access-b4c9b\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:02:45.120883 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.120686    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-token\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:02:45.120883 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.120702    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6a5522b-42fe-4030-bf46-852fc4382fe6-istio-podinfo\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:02:45.205273 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.205238    2562 generic.go:356] "Generic (PLEG): container finished" podID="b6a5522b-42fe-4030-bf46-852fc4382fe6" containerID="a3c1d0e9b3d93fc49b2daeeff5e6cd0d195d236545c2a1bf160403b815d93f20" exitCode=0
Sep 14 08:02:45.205484 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.205293    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"
Sep 14 08:02:45.205484 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.205312    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr" event={"ID":"b6a5522b-42fe-4030-bf46-852fc4382fe6","Type":"ContainerDied","Data":"a3c1d0e9b3d93fc49b2daeeff5e6cd0d195d236545c2a1bf160403b815d93f20"}
Sep 14 08:02:45.205484 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.205385    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr" event={"ID":"b6a5522b-42fe-4030-bf46-852fc4382fe6","Type":"ContainerDied","Data":"dcd9332580bdf84b70c550c0f29e3b4551f2c351eb5d3d756095019327768352"}
Sep 14 08:02:45.205484 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.205408    2562 scope.go:117] "RemoveContainer" containerID="a3c1d0e9b3d93fc49b2daeeff5e6cd0d195d236545c2a1bf160403b815d93f20"
Sep 14 08:02:45.215248 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.215229    2562 scope.go:117] "RemoveContainer" containerID="4dce9a51d47869d58984f990f88a0df657dfa02be3c81128ebb7c15c71e5c22e"
Sep 14 08:02:45.223490 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.223471    2562 scope.go:117] "RemoveContainer" containerID="a3c1d0e9b3d93fc49b2daeeff5e6cd0d195d236545c2a1bf160403b815d93f20"
Sep 14 08:02:45.223788 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:02:45.223768    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3c1d0e9b3d93fc49b2daeeff5e6cd0d195d236545c2a1bf160403b815d93f20\": container with ID starting with a3c1d0e9b3d93fc49b2daeeff5e6cd0d195d236545c2a1bf160403b815d93f20 not found: ID does not exist" containerID="a3c1d0e9b3d93fc49b2daeeff5e6cd0d195d236545c2a1bf160403b815d93f20"
Sep 14 08:02:45.223864 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.223797    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3c1d0e9b3d93fc49b2daeeff5e6cd0d195d236545c2a1bf160403b815d93f20"} err="failed to get container status \"a3c1d0e9b3d93fc49b2daeeff5e6cd0d195d236545c2a1bf160403b815d93f20\": rpc error: code = NotFound desc = could not find container \"a3c1d0e9b3d93fc49b2daeeff5e6cd0d195d236545c2a1bf160403b815d93f20\": container with ID starting with a3c1d0e9b3d93fc49b2daeeff5e6cd0d195d236545c2a1bf160403b815d93f20 not found: ID does not exist"
Sep 14 08:02:45.223864 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.223816    2562 scope.go:117] "RemoveContainer" containerID="4dce9a51d47869d58984f990f88a0df657dfa02be3c81128ebb7c15c71e5c22e"
Sep 14 08:02:45.224040 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:02:45.224020    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dce9a51d47869d58984f990f88a0df657dfa02be3c81128ebb7c15c71e5c22e\": container with ID starting with 4dce9a51d47869d58984f990f88a0df657dfa02be3c81128ebb7c15c71e5c22e not found: ID does not exist" containerID="4dce9a51d47869d58984f990f88a0df657dfa02be3c81128ebb7c15c71e5c22e"
Sep 14 08:02:45.224136 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.224039    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dce9a51d47869d58984f990f88a0df657dfa02be3c81128ebb7c15c71e5c22e"} err="failed to get container status \"4dce9a51d47869d58984f990f88a0df657dfa02be3c81128ebb7c15c71e5c22e\": rpc error: code = NotFound desc = could not find container \"4dce9a51d47869d58984f990f88a0df657dfa02be3c81128ebb7c15c71e5c22e\": container with ID starting with 4dce9a51d47869d58984f990f88a0df657dfa02be3c81128ebb7c15c71e5c22e not found: ID does not exist"
Sep 14 08:02:45.226468 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.226446    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"]
Sep 14 08:02:45.230909 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.230887    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-fd181c19-openshift-default-599499bd6-6v8qr"]
Sep 14 08:02:45.349332 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:45.349304    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:46.114081 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:46.114047    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6a5522b-42fe-4030-bf46-852fc4382fe6" path="/var/lib/kubelet/pods/b6a5522b-42fe-4030-bf46-852fc4382fe6/volumes"
Sep 14 08:02:46.283461 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:46.283422    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:47.252369 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:47.252332    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:48.080851 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:48.080809    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:02:49.087165 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:02:49.087128    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:05.115230 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:05.115192    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:03:05.312594 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:05.312557    2562 generic.go:356] "Generic (PLEG): container finished" podID="75b332e7-a761-4796-8d34-3e2f4382c41e" containerID="c7cfd43b3b4a8fb0b59a060789602b6dd2884c56dd57fbca437f6821565cceb0" exitCode=0
Sep 14 08:03:05.312763 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:05.312629    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b" event={"ID":"75b332e7-a761-4796-8d34-3e2f4382c41e","Type":"ContainerDied","Data":"c7cfd43b3b4a8fb0b59a060789602b6dd2884c56dd57fbca437f6821565cceb0"}
Sep 14 08:03:05.313099 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:05.313085    2562 scope.go:117] "RemoveContainer" containerID="c7cfd43b3b4a8fb0b59a060789602b6dd2884c56dd57fbca437f6821565cceb0"
Sep 14 08:03:05.314280 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:05.314255    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:03:05.314344 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:05.314320    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:03:05.314381 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:05.314351    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:03:06.319630 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:06.319594    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b" event={"ID":"75b332e7-a761-4796-8d34-3e2f4382c41e","Type":"ContainerStarted","Data":"0538fc1ef198825be95258e099332f67395037d0879d11afd6839ff8008f0a4d"}
Sep 14 08:03:06.691897 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:06.691856    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:03:06.692103 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:06.692053    2562 patch_prober.go:28] interesting pod/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 14 08:03:06.692178 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:06.692123    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b" podUID="75b332e7-a761-4796-8d34-3e2f4382c41e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 14 08:03:06.881408 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:06.881363    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:07.406400 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:07.406362    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b931e88a-pfzl6"]
Sep 14 08:03:07.406793 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:07.406783    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6a5522b-42fe-4030-bf46-852fc4382fe6" containerName="istio-proxy"
Sep 14 08:03:07.406836 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:07.406795    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6a5522b-42fe-4030-bf46-852fc4382fe6" containerName="istio-proxy"
Sep 14 08:03:07.406836 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:07.406804    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6a5522b-42fe-4030-bf46-852fc4382fe6" containerName="istio-proxy"
Sep 14 08:03:07.406836 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:07.406809    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6a5522b-42fe-4030-bf46-852fc4382fe6" containerName="istio-proxy"
Sep 14 08:03:07.406934 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:07.406862    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6a5522b-42fe-4030-bf46-852fc4382fe6" containerName="istio-proxy"
Sep 14 08:03:07.410022 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:07.410004    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b931e88a-pfzl6"
Sep 14 08:03:07.412536 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:07.412517    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:03:07.412803 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:07.412787    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:03:07.419809 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:07.419780    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b931e88a-pfzl6"]
Sep 14 08:03:07.465124 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:07.465071    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f1f9a16-231a-4815-b85c-1ec19e7b74e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b931e88a-pfzl6\" (UID: \"6f1f9a16-231a-4815-b85c-1ec19e7b74e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b931e88a-pfzl6"
Sep 14 08:03:07.565983 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:07.565939    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f1f9a16-231a-4815-b85c-1ec19e7b74e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b931e88a-pfzl6\" (UID: \"6f1f9a16-231a-4815-b85c-1ec19e7b74e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b931e88a-pfzl6"
Sep 14 08:03:07.566861 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:07.566837    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f1f9a16-231a-4815-b85c-1ec19e7b74e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-b931e88a-pfzl6\" (UID: \"6f1f9a16-231a-4815-b85c-1ec19e7b74e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b931e88a-pfzl6"
Sep 14 08:03:07.691558 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:07.691529    2562 patch_prober.go:28] interesting pod/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 14 08:03:07.691726 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:07.691596    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b" podUID="75b332e7-a761-4796-8d34-3e2f4382c41e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 14 08:03:07.721769 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:07.721736    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b931e88a-pfzl6"
Sep 14 08:03:08.026813 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.026616    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b931e88a-pfzl6"]
Sep 14 08:03:08.029409 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:03:08.029379    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f1f9a16_231a_4815_b85c_1ec19e7b74e9.slice/crio-cccdb1882fc46579d3ed04f3ea8582fd6c8b5af2f1b64e808fc8d9f49724efd7 WatchSource:0}: Error finding container cccdb1882fc46579d3ed04f3ea8582fd6c8b5af2f1b64e808fc8d9f49724efd7: Status 404 returned error can't find the container with id cccdb1882fc46579d3ed04f3ea8582fd6c8b5af2f1b64e808fc8d9f49724efd7
Sep 14 08:03:08.033673 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.033650    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:08.331447 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.331358    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b931e88a-pfzl6" event={"ID":"6f1f9a16-231a-4815-b85c-1ec19e7b74e9","Type":"ContainerStarted","Data":"cccdb1882fc46579d3ed04f3ea8582fd6c8b5af2f1b64e808fc8d9f49724efd7"}
Sep 14 08:03:08.692112 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.692077    2562 patch_prober.go:28] interesting pod/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 14 08:03:08.692537 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.692136    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b" podUID="75b332e7-a761-4796-8d34-3e2f4382c41e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 14 08:03:08.814209 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.814168    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"]
Sep 14 08:03:08.814716 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.814698    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6a5522b-42fe-4030-bf46-852fc4382fe6" containerName="istio-proxy"
Sep 14 08:03:08.817999 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.817978    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:08.820454 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.820429    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-22b517\"/\"default-dockercfg-97qfd\""
Sep 14 08:03:08.820595 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.820430    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-22b517\"/\"openshift-service-ca.crt\""
Sep 14 08:03:08.821033 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.821012    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-22b517\"/\"kube-root-ca.crt\""
Sep 14 08:03:08.821087 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.821080    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-22b517\"/\"autores-model-22b517-kserve-self-signed-certs\""
Sep 14 08:03:08.831142 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.831115    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"]
Sep 14 08:03:08.865526 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.865302    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:08.876961 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.876920    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-home\") pod \"autores-model-22b517-kserve-67f57cc547-94lwj\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") " pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:08.877132 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.876974    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-tmp-dir\") pod \"autores-model-22b517-kserve-67f57cc547-94lwj\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") " pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:08.877132 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.876998    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-dshm\") pod \"autores-model-22b517-kserve-67f57cc547-94lwj\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") " pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:08.877132 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.877039    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8fsp\" (UniqueName: \"kubernetes.io/projected/d7f86840-9e6a-403a-a49e-56cc7f1b932d-kube-api-access-x8fsp\") pod \"autores-model-22b517-kserve-67f57cc547-94lwj\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") " pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:08.877132 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.877066    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7f86840-9e6a-403a-a49e-56cc7f1b932d-tls-certs\") pod \"autores-model-22b517-kserve-67f57cc547-94lwj\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") " pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:08.877273 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.877137    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-model-cache\") pod \"autores-model-22b517-kserve-67f57cc547-94lwj\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") " pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:08.979657 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.977880    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-dshm\") pod \"autores-model-22b517-kserve-67f57cc547-94lwj\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") " pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:08.979657 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.977936    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8fsp\" (UniqueName: \"kubernetes.io/projected/d7f86840-9e6a-403a-a49e-56cc7f1b932d-kube-api-access-x8fsp\") pod \"autores-model-22b517-kserve-67f57cc547-94lwj\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") " pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:08.979657 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.977972    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7f86840-9e6a-403a-a49e-56cc7f1b932d-tls-certs\") pod \"autores-model-22b517-kserve-67f57cc547-94lwj\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") " pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:08.979657 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.978007    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-model-cache\") pod \"autores-model-22b517-kserve-67f57cc547-94lwj\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") " pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:08.979657 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.978179    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-home\") pod \"autores-model-22b517-kserve-67f57cc547-94lwj\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") " pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:08.979657 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.978235    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-tmp-dir\") pod \"autores-model-22b517-kserve-67f57cc547-94lwj\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") " pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:08.979657 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.978848    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-tmp-dir\") pod \"autores-model-22b517-kserve-67f57cc547-94lwj\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") " pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:08.979657 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.979212    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-model-cache\") pod \"autores-model-22b517-kserve-67f57cc547-94lwj\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") " pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:08.979657 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.979610    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-home\") pod \"autores-model-22b517-kserve-67f57cc547-94lwj\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") " pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:08.982617 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.982328    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7f86840-9e6a-403a-a49e-56cc7f1b932d-tls-certs\") pod \"autores-model-22b517-kserve-67f57cc547-94lwj\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") " pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:08.984593 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.984558    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-dshm\") pod \"autores-model-22b517-kserve-67f57cc547-94lwj\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") " pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:08.989860 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:08.989805    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8fsp\" (UniqueName: \"kubernetes.io/projected/d7f86840-9e6a-403a-a49e-56cc7f1b932d-kube-api-access-x8fsp\") pod \"autores-model-22b517-kserve-67f57cc547-94lwj\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") " pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:09.127205 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:09.127170    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:09.368623 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:09.368579    2562 generic.go:356] "Generic (PLEG): container finished" podID="6f1f9a16-231a-4815-b85c-1ec19e7b74e9" containerID="24b8d99be711ebb8c6d2f2b5d3d9d50a03e690326fc3ad9895cfed949844bdd6" exitCode=0
Sep 14 08:03:09.368836 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:09.368674    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b931e88a-pfzl6" event={"ID":"6f1f9a16-231a-4815-b85c-1ec19e7b74e9","Type":"ContainerDied","Data":"24b8d99be711ebb8c6d2f2b5d3d9d50a03e690326fc3ad9895cfed949844bdd6"}
Sep 14 08:03:09.432599 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:09.432556    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"]
Sep 14 08:03:09.433054 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:03:09.433026    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd7f86840_9e6a_403a_a49e_56cc7f1b932d.slice/crio-24c4ac3325ec27b2099dd90cc4f8707231af631b6e89424bfcf1646dbc72d149 WatchSource:0}: Error finding container 24c4ac3325ec27b2099dd90cc4f8707231af631b6e89424bfcf1646dbc72d149: Status 404 returned error can't find the container with id 24c4ac3325ec27b2099dd90cc4f8707231af631b6e89424bfcf1646dbc72d149
Sep 14 08:03:09.436848 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:09.436823    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:09.692218 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:09.692168    2562 patch_prober.go:28] interesting pod/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 14 08:03:09.692622 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:09.692237    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b" podUID="75b332e7-a761-4796-8d34-3e2f4382c41e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 14 08:03:10.375685 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:10.375601    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj" event={"ID":"d7f86840-9e6a-403a-a49e-56cc7f1b932d","Type":"ContainerStarted","Data":"71383ab79037d06f2f6e80bd1b396ba82c51f51514b4b30a73f6b7d22e6a9952"}
Sep 14 08:03:10.375685 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:10.375649    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj" event={"ID":"d7f86840-9e6a-403a-a49e-56cc7f1b932d","Type":"ContainerStarted","Data":"24c4ac3325ec27b2099dd90cc4f8707231af631b6e89424bfcf1646dbc72d149"}
Sep 14 08:03:10.387521 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:10.387463    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:10.534246 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:10.532884    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b931e88a-pfzl6"
Sep 14 08:03:10.562525 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:10.559185    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj" podStartSLOduration=2.559159905 podStartE2EDuration="2.559159905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:03:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:03:10.417285841 +0000 UTC m=+892.873735847" watchObservedRunningTime="2026-09-14 08:03:10.559159905 +0000 UTC m=+893.015609899"
Sep 14 08:03:10.594238 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:10.594200    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f1f9a16-231a-4815-b85c-1ec19e7b74e9-maas-api-service-ca\") pod \"6f1f9a16-231a-4815-b85c-1ec19e7b74e9\" (UID: \"6f1f9a16-231a-4815-b85c-1ec19e7b74e9\") "
Sep 14 08:03:10.594565 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:10.594540    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f1f9a16-231a-4815-b85c-1ec19e7b74e9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6f1f9a16-231a-4815-b85c-1ec19e7b74e9" (UID: "6f1f9a16-231a-4815-b85c-1ec19e7b74e9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:03:10.594662 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:10.594631    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6f1f9a16-231a-4815-b85c-1ec19e7b74e9-maas-api-service-ca\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:03:10.692203 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:10.692167    2562 patch_prober.go:28] interesting pod/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 14 08:03:10.692388 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:10.692238    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b" podUID="75b332e7-a761-4796-8d34-3e2f4382c41e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 14 08:03:11.384412 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:11.384182    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b931e88a-pfzl6" event={"ID":"6f1f9a16-231a-4815-b85c-1ec19e7b74e9","Type":"ContainerDied","Data":"cccdb1882fc46579d3ed04f3ea8582fd6c8b5af2f1b64e808fc8d9f49724efd7"}
Sep 14 08:03:11.384412 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:11.384212    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b931e88a-pfzl6"
Sep 14 08:03:11.384412 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:11.384252    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cccdb1882fc46579d3ed04f3ea8582fd6c8b5af2f1b64e808fc8d9f49724efd7"
Sep 14 08:03:11.696475 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:11.696441    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:03:11.696979 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:11.696855    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:03:11.697426 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:11.697410    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:03:11.809938 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:11.809908    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:13.056721 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:13.056675    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:14.037823 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:14.037789    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:15.065315 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:15.065275    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:16.000837 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:16.000792    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:16.102304 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:16.102262    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:16.208128 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:16.208095    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:16.402607 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:16.402531    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b931e88a-pfzl6"]
Sep 14 08:03:16.416240 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:16.416208    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-b931e88a-pfzl6"]
Sep 14 08:03:16.532413 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:16.532380    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"]
Sep 14 08:03:16.532652 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:16.532621    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b" podUID="75b332e7-a761-4796-8d34-3e2f4382c41e" containerName="istio-proxy" containerID="cri-o://0538fc1ef198825be95258e099332f67395037d0879d11afd6839ff8008f0a4d" gracePeriod=30
Sep 14 08:03:17.450707 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:17.450674    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:17.562085 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:17.562052    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:18.113828 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:18.113791    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f1f9a16-231a-4815-b85c-1ec19e7b74e9" path="/var/lib/kubelet/pods/6f1f9a16-231a-4815-b85c-1ec19e7b74e9/volumes"
Sep 14 08:03:18.115184 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:18.115156    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:03:18.260109 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:18.260015    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:18.998876 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:18.998848    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:19.127314 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:19.127272    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:19.127314 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:19.127322    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:19.139815 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:19.139786    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:19.443990 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:19.443961    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:03:19.629678 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:19.629646    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:20.781431 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:20.781399    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:21.785488 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.785461    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:03:21.913210 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.913117    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82w9h\" (UniqueName: \"kubernetes.io/projected/75b332e7-a761-4796-8d34-3e2f4382c41e-kube-api-access-82w9h\") pod \"75b332e7-a761-4796-8d34-3e2f4382c41e\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") "
Sep 14 08:03:21.913210 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.913173    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-credential-socket\") pod \"75b332e7-a761-4796-8d34-3e2f4382c41e\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") "
Sep 14 08:03:21.913449 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.913214    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-workload-certs\") pod \"75b332e7-a761-4796-8d34-3e2f4382c41e\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") "
Sep 14 08:03:21.913449 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.913231    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-token\") pod \"75b332e7-a761-4796-8d34-3e2f4382c41e\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") "
Sep 14 08:03:21.913449 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.913261    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-workload-socket\") pod \"75b332e7-a761-4796-8d34-3e2f4382c41e\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") "
Sep 14 08:03:21.913449 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.913288    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75b332e7-a761-4796-8d34-3e2f4382c41e-istiod-ca-cert\") pod \"75b332e7-a761-4796-8d34-3e2f4382c41e\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") "
Sep 14 08:03:21.913449 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.913348    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-data\") pod \"75b332e7-a761-4796-8d34-3e2f4382c41e\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") "
Sep 14 08:03:21.913449 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.913400    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-envoy\") pod \"75b332e7-a761-4796-8d34-3e2f4382c41e\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") "
Sep 14 08:03:21.913797 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.913482    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-podinfo\") pod \"75b332e7-a761-4796-8d34-3e2f4382c41e\" (UID: \"75b332e7-a761-4796-8d34-3e2f4382c41e\") "
Sep 14 08:03:21.913797 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.913585    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "75b332e7-a761-4796-8d34-3e2f4382c41e" (UID: "75b332e7-a761-4796-8d34-3e2f4382c41e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:03:21.913797 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.913602    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "75b332e7-a761-4796-8d34-3e2f4382c41e" (UID: "75b332e7-a761-4796-8d34-3e2f4382c41e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:03:21.913797 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.913617    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "75b332e7-a761-4796-8d34-3e2f4382c41e" (UID: "75b332e7-a761-4796-8d34-3e2f4382c41e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:03:21.913797 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.913737    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-data" (OuterVolumeSpecName: "istio-data") pod "75b332e7-a761-4796-8d34-3e2f4382c41e" (UID: "75b332e7-a761-4796-8d34-3e2f4382c41e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:03:21.914063 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.913936    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75b332e7-a761-4796-8d34-3e2f4382c41e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "75b332e7-a761-4796-8d34-3e2f4382c41e" (UID: "75b332e7-a761-4796-8d34-3e2f4382c41e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:03:21.914063 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.913987    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-credential-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:03:21.914063 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.914006    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-workload-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:03:21.914063 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.914021    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-workload-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:03:21.914063 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.914033    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-data\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:03:21.915579 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.915545    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75b332e7-a761-4796-8d34-3e2f4382c41e-kube-api-access-82w9h" (OuterVolumeSpecName: "kube-api-access-82w9h") pod "75b332e7-a761-4796-8d34-3e2f4382c41e" (UID: "75b332e7-a761-4796-8d34-3e2f4382c41e"). InnerVolumeSpecName "kube-api-access-82w9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:03:21.915717 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.915665    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "75b332e7-a761-4796-8d34-3e2f4382c41e" (UID: "75b332e7-a761-4796-8d34-3e2f4382c41e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:03:21.915895 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.915874    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-token" (OuterVolumeSpecName: "istio-token") pod "75b332e7-a761-4796-8d34-3e2f4382c41e" (UID: "75b332e7-a761-4796-8d34-3e2f4382c41e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:03:21.915972 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.915919    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "75b332e7-a761-4796-8d34-3e2f4382c41e" (UID: "75b332e7-a761-4796-8d34-3e2f4382c41e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:03:21.961861 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:21.961825    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:22.015012 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:22.014919    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-token\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:03:22.015012 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:22.014963    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75b332e7-a761-4796-8d34-3e2f4382c41e-istiod-ca-cert\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:03:22.015263 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:22.015029    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-envoy\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:03:22.015263 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:22.015050    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75b332e7-a761-4796-8d34-3e2f4382c41e-istio-podinfo\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:03:22.015263 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:22.015062    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82w9h\" (UniqueName: \"kubernetes.io/projected/75b332e7-a761-4796-8d34-3e2f4382c41e-kube-api-access-82w9h\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:03:22.448335 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:22.448300    2562 generic.go:356] "Generic (PLEG): container finished" podID="75b332e7-a761-4796-8d34-3e2f4382c41e" containerID="0538fc1ef198825be95258e099332f67395037d0879d11afd6839ff8008f0a4d" exitCode=0
Sep 14 08:03:22.448552 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:22.448348    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"
Sep 14 08:03:22.448552 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:22.448388    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b" event={"ID":"75b332e7-a761-4796-8d34-3e2f4382c41e","Type":"ContainerDied","Data":"0538fc1ef198825be95258e099332f67395037d0879d11afd6839ff8008f0a4d"}
Sep 14 08:03:22.448552 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:22.448442    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b" event={"ID":"75b332e7-a761-4796-8d34-3e2f4382c41e","Type":"ContainerDied","Data":"a88ef4d89da23b6714b6a5bc7b978c26bf29aa8eadf0a2f07cabcc72fa0a2260"}
Sep 14 08:03:22.448552 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:22.448465    2562 scope.go:117] "RemoveContainer" containerID="0538fc1ef198825be95258e099332f67395037d0879d11afd6839ff8008f0a4d"
Sep 14 08:03:22.457135 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:22.457118    2562 scope.go:117] "RemoveContainer" containerID="c7cfd43b3b4a8fb0b59a060789602b6dd2884c56dd57fbca437f6821565cceb0"
Sep 14 08:03:22.465874 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:22.465853    2562 scope.go:117] "RemoveContainer" containerID="0538fc1ef198825be95258e099332f67395037d0879d11afd6839ff8008f0a4d"
Sep 14 08:03:22.466239 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:03:22.466218    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0538fc1ef198825be95258e099332f67395037d0879d11afd6839ff8008f0a4d\": container with ID starting with 0538fc1ef198825be95258e099332f67395037d0879d11afd6839ff8008f0a4d not found: ID does not exist" containerID="0538fc1ef198825be95258e099332f67395037d0879d11afd6839ff8008f0a4d"
Sep 14 08:03:22.466324 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:22.466250    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0538fc1ef198825be95258e099332f67395037d0879d11afd6839ff8008f0a4d"} err="failed to get container status \"0538fc1ef198825be95258e099332f67395037d0879d11afd6839ff8008f0a4d\": rpc error: code = NotFound desc = could not find container \"0538fc1ef198825be95258e099332f67395037d0879d11afd6839ff8008f0a4d\": container with ID starting with 0538fc1ef198825be95258e099332f67395037d0879d11afd6839ff8008f0a4d not found: ID does not exist"
Sep 14 08:03:22.466324 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:22.466272    2562 scope.go:117] "RemoveContainer" containerID="c7cfd43b3b4a8fb0b59a060789602b6dd2884c56dd57fbca437f6821565cceb0"
Sep 14 08:03:22.466607 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:03:22.466582    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7cfd43b3b4a8fb0b59a060789602b6dd2884c56dd57fbca437f6821565cceb0\": container with ID starting with c7cfd43b3b4a8fb0b59a060789602b6dd2884c56dd57fbca437f6821565cceb0 not found: ID does not exist" containerID="c7cfd43b3b4a8fb0b59a060789602b6dd2884c56dd57fbca437f6821565cceb0"
Sep 14 08:03:22.466740 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:22.466612    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7cfd43b3b4a8fb0b59a060789602b6dd2884c56dd57fbca437f6821565cceb0"} err="failed to get container status \"c7cfd43b3b4a8fb0b59a060789602b6dd2884c56dd57fbca437f6821565cceb0\": rpc error: code = NotFound desc = could not find container \"c7cfd43b3b4a8fb0b59a060789602b6dd2884c56dd57fbca437f6821565cceb0\": container with ID starting with c7cfd43b3b4a8fb0b59a060789602b6dd2884c56dd57fbca437f6821565cceb0 not found: ID does not exist"
Sep 14 08:03:22.484469 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:22.484430    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"]
Sep 14 08:03:22.486702 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:22.486675    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-b931e88a-openshift-default-59c96d6d6d-zsb7b"]
Sep 14 08:03:22.917211 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:22.917123    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:24.112765 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:24.112732    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75b332e7-a761-4796-8d34-3e2f4382c41e" path="/var/lib/kubelet/pods/75b332e7-a761-4796-8d34-3e2f4382c41e/volumes"
Sep 14 08:03:24.458748 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:24.458714    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:26.212463 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:26.212421    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:26.309924 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:26.309888    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:26.419913 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:26.419886    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:26.523941 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:26.523848    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:26.939906 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:26.939871    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:33.123909 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:33.123872    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:03:42.719318 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:42.719279    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-22b517-wnq9g"]
Sep 14 08:03:42.719833 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:42.719705    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75b332e7-a761-4796-8d34-3e2f4382c41e" containerName="istio-proxy"
Sep 14 08:03:42.719833 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:42.719717    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="75b332e7-a761-4796-8d34-3e2f4382c41e" containerName="istio-proxy"
Sep 14 08:03:42.719833 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:42.719739    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f1f9a16-231a-4815-b85c-1ec19e7b74e9" containerName="revoke-keys"
Sep 14 08:03:42.719833 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:42.719745    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f1f9a16-231a-4815-b85c-1ec19e7b74e9" containerName="revoke-keys"
Sep 14 08:03:42.719833 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:42.719803    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="75b332e7-a761-4796-8d34-3e2f4382c41e" containerName="istio-proxy"
Sep 14 08:03:42.719833 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:42.719810    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f1f9a16-231a-4815-b85c-1ec19e7b74e9" containerName="revoke-keys"
Sep 14 08:03:42.724520 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:42.724484    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-22b517-wnq9g"
Sep 14 08:03:42.727479 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:42.727429    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:03:42.727693 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:42.727435    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:03:42.729974 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:42.729949    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-22b517-wnq9g"]
Sep 14 08:03:42.834587 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:42.834538    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fae3a1cc-9200-4699-b669-49217cb8a1dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-22b517-wnq9g\" (UID: \"fae3a1cc-9200-4699-b669-49217cb8a1dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-22b517-wnq9g"
Sep 14 08:03:42.935439 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:42.935397    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fae3a1cc-9200-4699-b669-49217cb8a1dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-22b517-wnq9g\" (UID: \"fae3a1cc-9200-4699-b669-49217cb8a1dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-22b517-wnq9g"
Sep 14 08:03:42.936056 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:42.936034    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fae3a1cc-9200-4699-b669-49217cb8a1dd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-22b517-wnq9g\" (UID: \"fae3a1cc-9200-4699-b669-49217cb8a1dd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-22b517-wnq9g"
Sep 14 08:03:42.985150 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:42.985056    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:43.038102 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:43.038067    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-22b517-wnq9g"
Sep 14 08:03:43.319281 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:43.319256    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-22b517-wnq9g"]
Sep 14 08:03:43.321579 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:03:43.321548    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfae3a1cc_9200_4699_b669_49217cb8a1dd.slice/crio-f33bd93d7fbdb0da77c0c2bb75f8aae662d637bf1f03000b9122ae93948032c1 WatchSource:0}: Error finding container f33bd93d7fbdb0da77c0c2bb75f8aae662d637bf1f03000b9122ae93948032c1: Status 404 returned error can't find the container with id f33bd93d7fbdb0da77c0c2bb75f8aae662d637bf1f03000b9122ae93948032c1
Sep 14 08:03:43.560388 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:43.560355    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-22b517-wnq9g" event={"ID":"fae3a1cc-9200-4699-b669-49217cb8a1dd","Type":"ContainerStarted","Data":"f33bd93d7fbdb0da77c0c2bb75f8aae662d637bf1f03000b9122ae93948032c1"}
Sep 14 08:03:44.567002 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:44.566964    2562 generic.go:356] "Generic (PLEG): container finished" podID="fae3a1cc-9200-4699-b669-49217cb8a1dd" containerID="073889c23e1015c7fe2a1235341734c96769ad55dd4a03acbfa11a15b83d4e36" exitCode=0
Sep 14 08:03:44.567433 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:44.567047    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-22b517-wnq9g" event={"ID":"fae3a1cc-9200-4699-b669-49217cb8a1dd","Type":"ContainerDied","Data":"073889c23e1015c7fe2a1235341734c96769ad55dd4a03acbfa11a15b83d4e36"}
Sep 14 08:03:44.692133 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:44.692096    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:44.793108 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:44.793074    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:45.472083 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:45.472042    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:45.707196 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:45.707170    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-22b517-wnq9g"
Sep 14 08:03:45.765941 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:45.765861    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fae3a1cc-9200-4699-b669-49217cb8a1dd-maas-api-service-ca\") pod \"fae3a1cc-9200-4699-b669-49217cb8a1dd\" (UID: \"fae3a1cc-9200-4699-b669-49217cb8a1dd\") "
Sep 14 08:03:45.766218 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:45.766195    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fae3a1cc-9200-4699-b669-49217cb8a1dd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fae3a1cc-9200-4699-b669-49217cb8a1dd" (UID: "fae3a1cc-9200-4699-b669-49217cb8a1dd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:03:45.776597 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:45.776569    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:45.867433 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:45.867395    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fae3a1cc-9200-4699-b669-49217cb8a1dd-maas-api-service-ca\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:03:46.496518 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:46.496449    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:46.580367 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:46.580340    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-22b517-wnq9g"
Sep 14 08:03:46.580367 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:46.580360    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-22b517-wnq9g" event={"ID":"fae3a1cc-9200-4699-b669-49217cb8a1dd","Type":"ContainerDied","Data":"f33bd93d7fbdb0da77c0c2bb75f8aae662d637bf1f03000b9122ae93948032c1"}
Sep 14 08:03:46.580615 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:46.580401    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f33bd93d7fbdb0da77c0c2bb75f8aae662d637bf1f03000b9122ae93948032c1"
Sep 14 08:03:47.316152 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:47.316116    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:48.115230 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:48.115198    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:52.861757 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:52.861710    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:53.486244 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:53.486210    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:53.558681 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:53.558649    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:53.621558 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:53.621522    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:53.723224 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:53.723185    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:53.742029 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:53.741946    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-22b517-wnq9g"]
Sep 14 08:03:53.745081 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:53.745056    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-22b517-wnq9g"]
Sep 14 08:03:53.843905 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:53.843831    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:54.098220 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:54.098099    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:54.113127 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:54.113095    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fae3a1cc-9200-4699-b669-49217cb8a1dd" path="/var/lib/kubelet/pods/fae3a1cc-9200-4699-b669-49217cb8a1dd/volumes"
Sep 14 08:03:54.240145 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:54.240094    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:55.043207 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:55.043166    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:55.252482 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:55.252441    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:55.485682 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:55.485647    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:56.041395 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:56.041358    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:56.807494 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:56.807448    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:57.746343 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:57.746307    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:58.671002 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:58.670963    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:59.045641 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:59.045605    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"]
Sep 14 08:03:59.072994 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:59.072953    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:59.208984 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:59.208947    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:03:59.556984 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:59.556949    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:03:59.775461 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:03:59.775422    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:00.330658 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:00.330626    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:00.680125 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:00.680025    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:00.742870 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:00.742834    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:00.808526 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:00.808459    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:00.908931 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:00.908891    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:06.298443 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:06.298405    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:06.389116 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:06.389081    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:06.654969 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:06.654883    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:06.765160 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:06.765124    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:07.598892 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:07.598858    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:08.610080 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:08.610024    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:09.445607 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:09.445555    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:12.987268 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:12.987220    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:13.087843 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:13.087812    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:13.241634 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:13.241552    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"]
Sep 14 08:04:13.241797 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:13.241776    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp" podUID="633c40e2-8c81-4de3-b01a-7dfe48f1c292" containerName="istio-proxy" containerID="cri-o://46ec19b308308b5b4b05ddb361566a5b7f0833f86b64549b2a04f9db05df8892" gracePeriod=30
Sep 14 08:04:13.248105 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:13.248070    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:14.064596 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.064557    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj" podUID="d7f86840-9e6a-403a-a49e-56cc7f1b932d" containerName="main" containerID="cri-o://71383ab79037d06f2f6e80bd1b396ba82c51f51514b4b30a73f6b7d22e6a9952" gracePeriod=45
Sep 14 08:04:14.312914 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.312886    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:04:14.356041 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.355960    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-dshm\") pod \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") "
Sep 14 08:04:14.356041 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.356007    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-home\") pod \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") "
Sep 14 08:04:14.356266 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.356051    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-tmp-dir\") pod \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") "
Sep 14 08:04:14.356266 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.356100    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7f86840-9e6a-403a-a49e-56cc7f1b932d-tls-certs\") pod \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") "
Sep 14 08:04:14.356266 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.356161    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-model-cache\") pod \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") "
Sep 14 08:04:14.356266 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.356183    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8fsp\" (UniqueName: \"kubernetes.io/projected/d7f86840-9e6a-403a-a49e-56cc7f1b932d-kube-api-access-x8fsp\") pod \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\" (UID: \"d7f86840-9e6a-403a-a49e-56cc7f1b932d\") "
Sep 14 08:04:14.356478 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.356303    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d7f86840-9e6a-403a-a49e-56cc7f1b932d" (UID: "d7f86840-9e6a-403a-a49e-56cc7f1b932d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:14.356478 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.356325    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-home" (OuterVolumeSpecName: "home") pod "d7f86840-9e6a-403a-a49e-56cc7f1b932d" (UID: "d7f86840-9e6a-403a-a49e-56cc7f1b932d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:14.356478 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.356411    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-model-cache" (OuterVolumeSpecName: "model-cache") pod "d7f86840-9e6a-403a-a49e-56cc7f1b932d" (UID: "d7f86840-9e6a-403a-a49e-56cc7f1b932d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:14.356656 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.356587    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-model-cache\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:14.356656 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.356608    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-home\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:14.356656 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.356622    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-tmp-dir\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:14.358363 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.358328    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7f86840-9e6a-403a-a49e-56cc7f1b932d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d7f86840-9e6a-403a-a49e-56cc7f1b932d" (UID: "d7f86840-9e6a-403a-a49e-56cc7f1b932d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:04:14.358493 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.358385    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-dshm" (OuterVolumeSpecName: "dshm") pod "d7f86840-9e6a-403a-a49e-56cc7f1b932d" (UID: "d7f86840-9e6a-403a-a49e-56cc7f1b932d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:14.358587 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.358490    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7f86840-9e6a-403a-a49e-56cc7f1b932d-kube-api-access-x8fsp" (OuterVolumeSpecName: "kube-api-access-x8fsp") pod "d7f86840-9e6a-403a-a49e-56cc7f1b932d" (UID: "d7f86840-9e6a-403a-a49e-56cc7f1b932d"). InnerVolumeSpecName "kube-api-access-x8fsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:04:14.457446 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.457404    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7f86840-9e6a-403a-a49e-56cc7f1b932d-tls-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:14.457446 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.457438    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8fsp\" (UniqueName: \"kubernetes.io/projected/d7f86840-9e6a-403a-a49e-56cc7f1b932d-kube-api-access-x8fsp\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:14.457446 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.457448    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7f86840-9e6a-403a-a49e-56cc7f1b932d-dshm\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:14.733715 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.733675    2562 generic.go:356] "Generic (PLEG): container finished" podID="d7f86840-9e6a-403a-a49e-56cc7f1b932d" containerID="71383ab79037d06f2f6e80bd1b396ba82c51f51514b4b30a73f6b7d22e6a9952" exitCode=0
Sep 14 08:04:14.733882 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.733722    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"
Sep 14 08:04:14.733882 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.733759    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj" event={"ID":"d7f86840-9e6a-403a-a49e-56cc7f1b932d","Type":"ContainerDied","Data":"71383ab79037d06f2f6e80bd1b396ba82c51f51514b4b30a73f6b7d22e6a9952"}
Sep 14 08:04:14.733882 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.733806    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj" event={"ID":"d7f86840-9e6a-403a-a49e-56cc7f1b932d","Type":"ContainerDied","Data":"24c4ac3325ec27b2099dd90cc4f8707231af631b6e89424bfcf1646dbc72d149"}
Sep 14 08:04:14.733882 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.733830    2562 scope.go:117] "RemoveContainer" containerID="71383ab79037d06f2f6e80bd1b396ba82c51f51514b4b30a73f6b7d22e6a9952"
Sep 14 08:04:14.742985 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.742965    2562 scope.go:117] "RemoveContainer" containerID="71383ab79037d06f2f6e80bd1b396ba82c51f51514b4b30a73f6b7d22e6a9952"
Sep 14 08:04:14.743276 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:04:14.743258    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71383ab79037d06f2f6e80bd1b396ba82c51f51514b4b30a73f6b7d22e6a9952\": container with ID starting with 71383ab79037d06f2f6e80bd1b396ba82c51f51514b4b30a73f6b7d22e6a9952 not found: ID does not exist" containerID="71383ab79037d06f2f6e80bd1b396ba82c51f51514b4b30a73f6b7d22e6a9952"
Sep 14 08:04:14.743315 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.743290    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71383ab79037d06f2f6e80bd1b396ba82c51f51514b4b30a73f6b7d22e6a9952"} err="failed to get container status \"71383ab79037d06f2f6e80bd1b396ba82c51f51514b4b30a73f6b7d22e6a9952\": rpc error: code = NotFound desc = could not find container \"71383ab79037d06f2f6e80bd1b396ba82c51f51514b4b30a73f6b7d22e6a9952\": container with ID starting with 71383ab79037d06f2f6e80bd1b396ba82c51f51514b4b30a73f6b7d22e6a9952 not found: ID does not exist"
Sep 14 08:04:14.754610 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.754588    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"]
Sep 14 08:04:14.757976 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:14.757956    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-22b517/autores-model-22b517-kserve-67f57cc547-94lwj"]
Sep 14 08:04:16.112618 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:16.112583    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7f86840-9e6a-403a-a49e-56cc7f1b932d" path="/var/lib/kubelet/pods/d7f86840-9e6a-403a-a49e-56cc7f1b932d/volumes"
Sep 14 08:04:16.759189 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:16.759156    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:18.026389 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.026351    2562 patch_prober.go:28] interesting pod/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 08:04:18.026744 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.026417    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp" podUID="633c40e2-8c81-4de3-b01a-7dfe48f1c292" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 08:04:18.497143 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.497116    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:04:18.599294 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.599207    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-envoy\") pod \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") "
Sep 14 08:04:18.599294 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.599245    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-podinfo\") pod \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") "
Sep 14 08:04:18.599294 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.599270    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-workload-socket\") pod \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") "
Sep 14 08:04:18.599620 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.599295    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istiod-ca-cert\") pod \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") "
Sep 14 08:04:18.599620 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.599349    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-workload-certs\") pod \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") "
Sep 14 08:04:18.599620 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.599384    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-data\") pod \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") "
Sep 14 08:04:18.599620 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.599421    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dgf4\" (UniqueName: \"kubernetes.io/projected/633c40e2-8c81-4de3-b01a-7dfe48f1c292-kube-api-access-4dgf4\") pod \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") "
Sep 14 08:04:18.599620 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.599535    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-token\") pod \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") "
Sep 14 08:04:18.599620 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.599594    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-credential-socket\") pod \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\" (UID: \"633c40e2-8c81-4de3-b01a-7dfe48f1c292\") "
Sep 14 08:04:18.599969 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.599635    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "633c40e2-8c81-4de3-b01a-7dfe48f1c292" (UID: "633c40e2-8c81-4de3-b01a-7dfe48f1c292"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:18.599969 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.599705    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "633c40e2-8c81-4de3-b01a-7dfe48f1c292" (UID: "633c40e2-8c81-4de3-b01a-7dfe48f1c292"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:18.599969 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.599738    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "633c40e2-8c81-4de3-b01a-7dfe48f1c292" (UID: "633c40e2-8c81-4de3-b01a-7dfe48f1c292"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:04:18.599969 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.599919    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-data" (OuterVolumeSpecName: "istio-data") pod "633c40e2-8c81-4de3-b01a-7dfe48f1c292" (UID: "633c40e2-8c81-4de3-b01a-7dfe48f1c292"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:18.599969 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.599940    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "633c40e2-8c81-4de3-b01a-7dfe48f1c292" (UID: "633c40e2-8c81-4de3-b01a-7dfe48f1c292"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:18.600155 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.599998    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-workload-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:18.600155 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.600019    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istiod-ca-cert\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:18.600155 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.600030    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-workload-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:18.601529 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.601485    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "633c40e2-8c81-4de3-b01a-7dfe48f1c292" (UID: "633c40e2-8c81-4de3-b01a-7dfe48f1c292"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:18.601705 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.601684    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/633c40e2-8c81-4de3-b01a-7dfe48f1c292-kube-api-access-4dgf4" (OuterVolumeSpecName: "kube-api-access-4dgf4") pod "633c40e2-8c81-4de3-b01a-7dfe48f1c292" (UID: "633c40e2-8c81-4de3-b01a-7dfe48f1c292"). InnerVolumeSpecName "kube-api-access-4dgf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:04:18.601985 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.601966    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-token" (OuterVolumeSpecName: "istio-token") pod "633c40e2-8c81-4de3-b01a-7dfe48f1c292" (UID: "633c40e2-8c81-4de3-b01a-7dfe48f1c292"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:04:18.602030 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.601965    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "633c40e2-8c81-4de3-b01a-7dfe48f1c292" (UID: "633c40e2-8c81-4de3-b01a-7dfe48f1c292"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:04:18.701610 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.701571    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-envoy\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:18.701610 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.701603    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-podinfo\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:18.701610 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.701612    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-data\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:18.701610 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.701621    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dgf4\" (UniqueName: \"kubernetes.io/projected/633c40e2-8c81-4de3-b01a-7dfe48f1c292-kube-api-access-4dgf4\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:18.701888 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.701630    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/633c40e2-8c81-4de3-b01a-7dfe48f1c292-istio-token\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:18.701888 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.701639    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/633c40e2-8c81-4de3-b01a-7dfe48f1c292-credential-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:18.757108 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.757070    2562 generic.go:356] "Generic (PLEG): container finished" podID="633c40e2-8c81-4de3-b01a-7dfe48f1c292" containerID="46ec19b308308b5b4b05ddb361566a5b7f0833f86b64549b2a04f9db05df8892" exitCode=0
Sep 14 08:04:18.757267 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.757129    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"
Sep 14 08:04:18.757267 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.757140    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp" event={"ID":"633c40e2-8c81-4de3-b01a-7dfe48f1c292","Type":"ContainerDied","Data":"46ec19b308308b5b4b05ddb361566a5b7f0833f86b64549b2a04f9db05df8892"}
Sep 14 08:04:18.757267 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.757193    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp" event={"ID":"633c40e2-8c81-4de3-b01a-7dfe48f1c292","Type":"ContainerDied","Data":"caf93fe823fde09421f88551e6bc4f530f11a635b55c56bad93be17efe90c514"}
Sep 14 08:04:18.757267 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.757223    2562 scope.go:117] "RemoveContainer" containerID="46ec19b308308b5b4b05ddb361566a5b7f0833f86b64549b2a04f9db05df8892"
Sep 14 08:04:18.766920 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.766895    2562 scope.go:117] "RemoveContainer" containerID="caa481dc1b81fb749f486e6e55ac9620fb0497ea0e98b33c7f9ad2d220f6c138"
Sep 14 08:04:18.774929 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.774907    2562 scope.go:117] "RemoveContainer" containerID="46ec19b308308b5b4b05ddb361566a5b7f0833f86b64549b2a04f9db05df8892"
Sep 14 08:04:18.775199 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:04:18.775180    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46ec19b308308b5b4b05ddb361566a5b7f0833f86b64549b2a04f9db05df8892\": container with ID starting with 46ec19b308308b5b4b05ddb361566a5b7f0833f86b64549b2a04f9db05df8892 not found: ID does not exist" containerID="46ec19b308308b5b4b05ddb361566a5b7f0833f86b64549b2a04f9db05df8892"
Sep 14 08:04:18.775244 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.775210    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46ec19b308308b5b4b05ddb361566a5b7f0833f86b64549b2a04f9db05df8892"} err="failed to get container status \"46ec19b308308b5b4b05ddb361566a5b7f0833f86b64549b2a04f9db05df8892\": rpc error: code = NotFound desc = could not find container \"46ec19b308308b5b4b05ddb361566a5b7f0833f86b64549b2a04f9db05df8892\": container with ID starting with 46ec19b308308b5b4b05ddb361566a5b7f0833f86b64549b2a04f9db05df8892 not found: ID does not exist"
Sep 14 08:04:18.775244 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.775228    2562 scope.go:117] "RemoveContainer" containerID="caa481dc1b81fb749f486e6e55ac9620fb0497ea0e98b33c7f9ad2d220f6c138"
Sep 14 08:04:18.775495 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:04:18.775478    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"caa481dc1b81fb749f486e6e55ac9620fb0497ea0e98b33c7f9ad2d220f6c138\": container with ID starting with caa481dc1b81fb749f486e6e55ac9620fb0497ea0e98b33c7f9ad2d220f6c138 not found: ID does not exist" containerID="caa481dc1b81fb749f486e6e55ac9620fb0497ea0e98b33c7f9ad2d220f6c138"
Sep 14 08:04:18.775585 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.775509    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"caa481dc1b81fb749f486e6e55ac9620fb0497ea0e98b33c7f9ad2d220f6c138"} err="failed to get container status \"caa481dc1b81fb749f486e6e55ac9620fb0497ea0e98b33c7f9ad2d220f6c138\": rpc error: code = NotFound desc = could not find container \"caa481dc1b81fb749f486e6e55ac9620fb0497ea0e98b33c7f9ad2d220f6c138\": container with ID starting with caa481dc1b81fb749f486e6e55ac9620fb0497ea0e98b33c7f9ad2d220f6c138 not found: ID does not exist"
Sep 14 08:04:18.787282 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.787250    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"]
Sep 14 08:04:18.789645 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:18.789624    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-814672-openshift-default-78996d8b6d-4ppjp"]
Sep 14 08:04:19.614314 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:19.614278    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:04:20.112362 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:20.112329    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="633c40e2-8c81-4de3-b01a-7dfe48f1c292" path="/var/lib/kubelet/pods/633c40e2-8c81-4de3-b01a-7dfe48f1c292/volumes"
Sep 14 08:04:24.427956 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:24.427916    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:24.530305 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:24.530270    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:24.632096 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:24.632061    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:25.094124 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:25.094083    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:25.337759 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:25.337722    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:25.449052 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:25.449022    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:25.570775 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:25.570739    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"]
Sep 14 08:04:25.571077 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:25.571048    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b" podUID="732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" containerName="istio-proxy" containerID="cri-o://d8a306ba1088c317f0132f3d138440d6b9695b79b15003720cc5966d41bd98a4" gracePeriod=30
Sep 14 08:04:25.592591 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:25.592555    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:26.038311 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.038275    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:26.325005 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.324928    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"]
Sep 14 08:04:26.325310 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.325298    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fae3a1cc-9200-4699-b669-49217cb8a1dd" containerName="revoke-keys"
Sep 14 08:04:26.325358 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.325312    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fae3a1cc-9200-4699-b669-49217cb8a1dd" containerName="revoke-keys"
Sep 14 08:04:26.325358 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.325324    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="633c40e2-8c81-4de3-b01a-7dfe48f1c292" containerName="istio-proxy"
Sep 14 08:04:26.325358 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.325332    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="633c40e2-8c81-4de3-b01a-7dfe48f1c292" containerName="istio-proxy"
Sep 14 08:04:26.325358 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.325340    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="633c40e2-8c81-4de3-b01a-7dfe48f1c292" containerName="istio-proxy"
Sep 14 08:04:26.325358 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.325346    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="633c40e2-8c81-4de3-b01a-7dfe48f1c292" containerName="istio-proxy"
Sep 14 08:04:26.325358 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.325351    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75b332e7-a761-4796-8d34-3e2f4382c41e" containerName="istio-proxy"
Sep 14 08:04:26.325358 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.325356    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="75b332e7-a761-4796-8d34-3e2f4382c41e" containerName="istio-proxy"
Sep 14 08:04:26.325584 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.325379    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7f86840-9e6a-403a-a49e-56cc7f1b932d" containerName="main"
Sep 14 08:04:26.325584 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.325385    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7f86840-9e6a-403a-a49e-56cc7f1b932d" containerName="main"
Sep 14 08:04:26.325584 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.325439    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="633c40e2-8c81-4de3-b01a-7dfe48f1c292" containerName="istio-proxy"
Sep 14 08:04:26.325584 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.325448    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7f86840-9e6a-403a-a49e-56cc7f1b932d" containerName="main"
Sep 14 08:04:26.325584 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.325458    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="75b332e7-a761-4796-8d34-3e2f4382c41e" containerName="istio-proxy"
Sep 14 08:04:26.325584 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.325464    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fae3a1cc-9200-4699-b669-49217cb8a1dd" containerName="revoke-keys"
Sep 14 08:04:26.330104 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.330086    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.332790 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.332764    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-73bd4a-openshift-default-dockercfg-cxv8x\""
Sep 14 08:04:26.341357 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.341326    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"]
Sep 14 08:04:26.344838 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.344811    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:26.477154 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.477117    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-credential-socket\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.477615 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.477230    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-data\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.477615 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.477278    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istiod-ca-cert\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.477615 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.477312    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-envoy\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.477615 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.477328    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-podinfo\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.477615 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.477395    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-token\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.477615 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.477433    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-workload-socket\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.477615 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.477481    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw86f\" (UniqueName: \"kubernetes.io/projected/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-kube-api-access-pw86f\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.477615 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.477558    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-workload-certs\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.579149 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.579057    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-data\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.579149 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.579100    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istiod-ca-cert\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.579149 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.579140    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-envoy\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.579433 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.579156    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-podinfo\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.579433 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.579178    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-token\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.579433 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.579194    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-workload-socket\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.579433 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.579226    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw86f\" (UniqueName: \"kubernetes.io/projected/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-kube-api-access-pw86f\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.579433 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.579277    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-workload-certs\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.579433 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.579321    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-credential-socket\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.579801 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.579575    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-data\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.580041 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.580015    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-workload-socket\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.580128 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.580055    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-workload-certs\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.580128 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.580065    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istiod-ca-cert\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.580128 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.580109    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-credential-socket\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.581630 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.581605    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-envoy\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.581792 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.581773    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-podinfo\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.587381 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.587356    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-token\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.587493 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.587459    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw86f\" (UniqueName: \"kubernetes.io/projected/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-kube-api-access-pw86f\") pod \"e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") " pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.641509 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.641471    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:26.768873 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.768838    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:26.930347 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.930317    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"]
Sep 14 08:04:26.932670 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:04:26.932646    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfef6e286_c65d_46bd_a85f_aa6eca82dbaa.slice/crio-749fbd4fc7c87137953a6345aece5cf8e542a0107c8fe5cd0ceda7fdadd2c080 WatchSource:0}: Error finding container 749fbd4fc7c87137953a6345aece5cf8e542a0107c8fe5cd0ceda7fdadd2c080: Status 404 returned error can't find the container with id 749fbd4fc7c87137953a6345aece5cf8e542a0107c8fe5cd0ceda7fdadd2c080
Sep 14 08:04:26.935071 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.935029    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:04:26.935166 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.935148    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:04:26.935207 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:26.935195    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:04:27.015170 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:27.015134    2562 patch_prober.go:28] interesting pod/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 08:04:27.015341 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:27.015194    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b" podUID="732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 08:04:27.803596 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:27.803554    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s" event={"ID":"fef6e286-c65d-46bd-a85f-aa6eca82dbaa","Type":"ContainerStarted","Data":"91bbf91cd2588c4d50f1ec86683f95b842e0a374c5ce2f7f75e045e890dcace4"}
Sep 14 08:04:27.804003 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:27.803613    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s" event={"ID":"fef6e286-c65d-46bd-a85f-aa6eca82dbaa","Type":"ContainerStarted","Data":"749fbd4fc7c87137953a6345aece5cf8e542a0107c8fe5cd0ceda7fdadd2c080"}
Sep 14 08:04:27.828793 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:27.828729    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s" podStartSLOduration=1.828710841 podStartE2EDuration="1.828710841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:04:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:04:27.827590173 +0000 UTC m=+970.284040171" watchObservedRunningTime="2026-09-14 08:04:27.828710841 +0000 UTC m=+970.285160834"
Sep 14 08:04:28.642372 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:28.642333    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:28.647368 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:28.647343    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:28.808909 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:28.808876    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:28.809897 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:28.809880    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:29.182477 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:29.182436    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:29.851375 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:29.851339    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:29.954433 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:29.954395    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:30.238265 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.238227    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:30.814382 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.814357    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:04:30.820045 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.820016    2562 generic.go:356] "Generic (PLEG): container finished" podID="732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" containerID="d8a306ba1088c317f0132f3d138440d6b9695b79b15003720cc5966d41bd98a4" exitCode=0
Sep 14 08:04:30.820181 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.820061    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"
Sep 14 08:04:30.820181 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.820103    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b" event={"ID":"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0","Type":"ContainerDied","Data":"d8a306ba1088c317f0132f3d138440d6b9695b79b15003720cc5966d41bd98a4"}
Sep 14 08:04:30.820181 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.820157    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b" event={"ID":"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0","Type":"ContainerDied","Data":"4400356fbf67559a8b962aedaf8b5c31a42ed8d6502ac42bb991df0ae8109195"}
Sep 14 08:04:30.820287 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.820189    2562 scope.go:117] "RemoveContainer" containerID="d8a306ba1088c317f0132f3d138440d6b9695b79b15003720cc5966d41bd98a4"
Sep 14 08:04:30.829244 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.829219    2562 scope.go:117] "RemoveContainer" containerID="0ca7ed0ae57e249787379aad80a05caf93212a1497baedcb6cce8badf3dd9bee"
Sep 14 08:04:30.839340 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.839316    2562 scope.go:117] "RemoveContainer" containerID="d8a306ba1088c317f0132f3d138440d6b9695b79b15003720cc5966d41bd98a4"
Sep 14 08:04:30.839775 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:04:30.839752    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8a306ba1088c317f0132f3d138440d6b9695b79b15003720cc5966d41bd98a4\": container with ID starting with d8a306ba1088c317f0132f3d138440d6b9695b79b15003720cc5966d41bd98a4 not found: ID does not exist" containerID="d8a306ba1088c317f0132f3d138440d6b9695b79b15003720cc5966d41bd98a4"
Sep 14 08:04:30.839866 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.839782    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8a306ba1088c317f0132f3d138440d6b9695b79b15003720cc5966d41bd98a4"} err="failed to get container status \"d8a306ba1088c317f0132f3d138440d6b9695b79b15003720cc5966d41bd98a4\": rpc error: code = NotFound desc = could not find container \"d8a306ba1088c317f0132f3d138440d6b9695b79b15003720cc5966d41bd98a4\": container with ID starting with d8a306ba1088c317f0132f3d138440d6b9695b79b15003720cc5966d41bd98a4 not found: ID does not exist"
Sep 14 08:04:30.839866 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.839801    2562 scope.go:117] "RemoveContainer" containerID="0ca7ed0ae57e249787379aad80a05caf93212a1497baedcb6cce8badf3dd9bee"
Sep 14 08:04:30.840122 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:04:30.840104    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ca7ed0ae57e249787379aad80a05caf93212a1497baedcb6cce8badf3dd9bee\": container with ID starting with 0ca7ed0ae57e249787379aad80a05caf93212a1497baedcb6cce8badf3dd9bee not found: ID does not exist" containerID="0ca7ed0ae57e249787379aad80a05caf93212a1497baedcb6cce8badf3dd9bee"
Sep 14 08:04:30.840187 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.840124    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ca7ed0ae57e249787379aad80a05caf93212a1497baedcb6cce8badf3dd9bee"} err="failed to get container status \"0ca7ed0ae57e249787379aad80a05caf93212a1497baedcb6cce8badf3dd9bee\": rpc error: code = NotFound desc = could not find container \"0ca7ed0ae57e249787379aad80a05caf93212a1497baedcb6cce8badf3dd9bee\": container with ID starting with 0ca7ed0ae57e249787379aad80a05caf93212a1497baedcb6cce8badf3dd9bee not found: ID does not exist"
Sep 14 08:04:30.924594 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.924560    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-envoy\") pod \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") "
Sep 14 08:04:30.925065 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.924607    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-workload-socket\") pod \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") "
Sep 14 08:04:30.925065 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.924645    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istiod-ca-cert\") pod \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") "
Sep 14 08:04:30.925065 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.924730    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-workload-certs\") pod \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") "
Sep 14 08:04:30.925065 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.924777    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-podinfo\") pod \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") "
Sep 14 08:04:30.925065 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.924811    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nphbg\" (UniqueName: \"kubernetes.io/projected/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-kube-api-access-nphbg\") pod \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") "
Sep 14 08:04:30.925065 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.924853    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-data\") pod \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") "
Sep 14 08:04:30.925065 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.924897    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-token\") pod \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") "
Sep 14 08:04:30.925065 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.924904    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" (UID: "732d49d6-cc23-4fcb-b5ff-7dad71ba89d0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:30.925065 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.924932    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-credential-socket\") pod \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\" (UID: \"732d49d6-cc23-4fcb-b5ff-7dad71ba89d0\") "
Sep 14 08:04:30.925525 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.925147    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" (UID: "732d49d6-cc23-4fcb-b5ff-7dad71ba89d0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:04:30.925525 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.925165    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-data" (OuterVolumeSpecName: "istio-data") pod "732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" (UID: "732d49d6-cc23-4fcb-b5ff-7dad71ba89d0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:30.925525 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.925299    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" (UID: "732d49d6-cc23-4fcb-b5ff-7dad71ba89d0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:30.925525 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.925371    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-workload-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:30.925525 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.925390    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-data\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:30.925525 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.925403    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-workload-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:30.925525 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.925403    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" (UID: "732d49d6-cc23-4fcb-b5ff-7dad71ba89d0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:30.925525 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.925416    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istiod-ca-cert\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:30.927024 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.926991    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" (UID: "732d49d6-cc23-4fcb-b5ff-7dad71ba89d0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:04:30.927024 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.926993    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-kube-api-access-nphbg" (OuterVolumeSpecName: "kube-api-access-nphbg") pod "732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" (UID: "732d49d6-cc23-4fcb-b5ff-7dad71ba89d0"). InnerVolumeSpecName "kube-api-access-nphbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:04:30.927223 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.927200    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" (UID: "732d49d6-cc23-4fcb-b5ff-7dad71ba89d0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:30.927393 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:30.927372    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-token" (OuterVolumeSpecName: "istio-token") pod "732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" (UID: "732d49d6-cc23-4fcb-b5ff-7dad71ba89d0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:04:31.026240 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:31.026202    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-podinfo\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:31.026240 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:31.026235    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nphbg\" (UniqueName: \"kubernetes.io/projected/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-kube-api-access-nphbg\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:31.026240 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:31.026248    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-token\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:31.026484 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:31.026261    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-credential-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:31.026484 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:31.026272    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0-istio-envoy\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:04:31.144391 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:31.144351    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"]
Sep 14 08:04:31.152541 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:31.152487    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-22b517-openshift-default-7b499f4984-kxs9b"]
Sep 14 08:04:31.203187 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:31.203149    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:31.781717 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:31.781679    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:31.882363 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:31.882329    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:31.983337 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:31.983298    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:32.113718 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:32.113637    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" path="/var/lib/kubelet/pods/732d49d6-cc23-4fcb-b5ff-7dad71ba89d0/volumes"
Sep 14 08:04:32.314894 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:32.314854    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:04:33.011721 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:33.011690    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:33.111395 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:33.111356    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:33.230447 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:33.230412    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:33.573125 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:33.573088    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:33.665922 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:33.665883    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:33.888437 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:33.888346    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:34.682361 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:34.682324    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:40.940000 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:40.939908    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:40.998612 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:40.998560    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:41.079544 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:41.079474    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:41.175340 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:41.175297    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:41.273430 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:41.273340    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:41.907876 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:41.907829    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:42.009136 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:42.009097    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:42.116948 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:42.116895    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:43.720413 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:43.720369    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:49.317569 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:49.317532    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:04:49.921022 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:49.920941    2562 generic.go:356] "Generic (PLEG): container finished" podID="fef6e286-c65d-46bd-a85f-aa6eca82dbaa" containerID="91bbf91cd2588c4d50f1ec86683f95b842e0a374c5ce2f7f75e045e890dcace4" exitCode=0
Sep 14 08:04:49.921022 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:49.920993    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s" event={"ID":"fef6e286-c65d-46bd-a85f-aa6eca82dbaa","Type":"ContainerDied","Data":"91bbf91cd2588c4d50f1ec86683f95b842e0a374c5ce2f7f75e045e890dcace4"}
Sep 14 08:04:49.921424 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:49.921403    2562 scope.go:117] "RemoveContainer" containerID="91bbf91cd2588c4d50f1ec86683f95b842e0a374c5ce2f7f75e045e890dcace4"
Sep 14 08:04:49.922612 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:49.922565    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:04:49.922751 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:49.922664    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:04:49.922751 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:49.922713    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:04:50.143235 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:50.143144    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:50.252756 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:50.252720    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:50.878423 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:50.878392    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:50.927792 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:50.927755    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s" event={"ID":"fef6e286-c65d-46bd-a85f-aa6eca82dbaa","Type":"ContainerStarted","Data":"42d0ebf246b5388c59de929e77c942f35dd572bc4117d90a8e5741f2d8613b79"}
Sep 14 08:04:51.642603 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:51.642559    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:51.643047 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:51.643016    2562 patch_prober.go:28] interesting pod/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 14 08:04:51.643277 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:51.643239    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s" podUID="fef6e286-c65d-46bd-a85f-aa6eca82dbaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 14 08:04:52.513330 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.513290    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"]
Sep 14 08:04:52.513748 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.513680    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" containerName="istio-proxy"
Sep 14 08:04:52.513748 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.513691    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" containerName="istio-proxy"
Sep 14 08:04:52.513748 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.513700    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" containerName="istio-proxy"
Sep 14 08:04:52.513748 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.513706    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" containerName="istio-proxy"
Sep 14 08:04:52.513886 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.513776    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" containerName="istio-proxy"
Sep 14 08:04:52.513886 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.513784    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="732d49d6-cc23-4fcb-b5ff-7dad71ba89d0" containerName="istio-proxy"
Sep 14 08:04:52.513886 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.513790    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="633c40e2-8c81-4de3-b01a-7dfe48f1c292" containerName="istio-proxy"
Sep 14 08:04:52.515757 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.515736    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:04:52.519152 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.519120    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-5ae4e6\"/\"test-model-5ae4e6-kserve-self-signed-certs\""
Sep 14 08:04:52.519316 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.519151    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-5ae4e6\"/\"openshift-service-ca.crt\""
Sep 14 08:04:52.519316 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.519151    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-5ae4e6\"/\"kube-root-ca.crt\""
Sep 14 08:04:52.519316 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.519255    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-5ae4e6\"/\"default-dockercfg-hsfh7\""
Sep 14 08:04:52.531944 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.531922    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"]
Sep 14 08:04:52.637567 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.637533    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:52.643523 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.642826    2562 patch_prober.go:28] interesting pod/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 14 08:04:52.643523 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.642895    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s" podUID="fef6e286-c65d-46bd-a85f-aa6eca82dbaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 14 08:04:52.651050 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.651016    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-tmp-dir\") pod \"test-model-5ae4e6-kserve-85c844bb57-mb8qr\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") " pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:04:52.651331 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.651312    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-model-cache\") pod \"test-model-5ae4e6-kserve-85c844bb57-mb8qr\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") " pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:04:52.651479 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.651462    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-home\") pod \"test-model-5ae4e6-kserve-85c844bb57-mb8qr\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") " pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:04:52.651645 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.651628    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d91878b-c89b-4d05-8efa-6836e3ab8872-tls-certs\") pod \"test-model-5ae4e6-kserve-85c844bb57-mb8qr\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") " pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:04:52.651773 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.651757    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4tkc\" (UniqueName: \"kubernetes.io/projected/6d91878b-c89b-4d05-8efa-6836e3ab8872-kube-api-access-j4tkc\") pod \"test-model-5ae4e6-kserve-85c844bb57-mb8qr\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") " pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:04:52.651955 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.651935    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-dshm\") pod \"test-model-5ae4e6-kserve-85c844bb57-mb8qr\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") " pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:04:52.753331 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.753284    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-dshm\") pod \"test-model-5ae4e6-kserve-85c844bb57-mb8qr\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") " pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:04:52.753562 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.753377    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-tmp-dir\") pod \"test-model-5ae4e6-kserve-85c844bb57-mb8qr\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") " pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:04:52.753562 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.753407    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-model-cache\") pod \"test-model-5ae4e6-kserve-85c844bb57-mb8qr\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") " pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:04:52.753562 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.753437    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-home\") pod \"test-model-5ae4e6-kserve-85c844bb57-mb8qr\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") " pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:04:52.753562 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.753479    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d91878b-c89b-4d05-8efa-6836e3ab8872-tls-certs\") pod \"test-model-5ae4e6-kserve-85c844bb57-mb8qr\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") " pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:04:52.753562 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.753533    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4tkc\" (UniqueName: \"kubernetes.io/projected/6d91878b-c89b-4d05-8efa-6836e3ab8872-kube-api-access-j4tkc\") pod \"test-model-5ae4e6-kserve-85c844bb57-mb8qr\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") " pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:04:52.753974 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.753940    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-model-cache\") pod \"test-model-5ae4e6-kserve-85c844bb57-mb8qr\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") " pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:04:52.754373 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.754326    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-tmp-dir\") pod \"test-model-5ae4e6-kserve-85c844bb57-mb8qr\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") " pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:04:52.754720 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.754700    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-home\") pod \"test-model-5ae4e6-kserve-85c844bb57-mb8qr\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") " pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:04:52.756919 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.756892    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d91878b-c89b-4d05-8efa-6836e3ab8872-tls-certs\") pod \"test-model-5ae4e6-kserve-85c844bb57-mb8qr\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") " pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:04:52.762001 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.761974    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-dshm\") pod \"test-model-5ae4e6-kserve-85c844bb57-mb8qr\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") " pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:04:52.778208 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.778142    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4tkc\" (UniqueName: \"kubernetes.io/projected/6d91878b-c89b-4d05-8efa-6836e3ab8872-kube-api-access-j4tkc\") pod \"test-model-5ae4e6-kserve-85c844bb57-mb8qr\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") " pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:04:52.824911 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:52.824869    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:04:53.126531 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:53.126487    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"]
Sep 14 08:04:53.128395 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:04:53.128367    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d91878b_c89b_4d05_8efa_6836e3ab8872.slice/crio-c87e92dee75895b8a24a59390d8b563e0ea17b16ec85940f781bc17c87f61c39 WatchSource:0}: Error finding container c87e92dee75895b8a24a59390d8b563e0ea17b16ec85940f781bc17c87f61c39: Status 404 returned error can't find the container with id c87e92dee75895b8a24a59390d8b563e0ea17b16ec85940f781bc17c87f61c39
Sep 14 08:04:53.642683 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:53.642640    2562 patch_prober.go:28] interesting pod/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 14 08:04:53.643178 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:53.642709    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s" podUID="fef6e286-c65d-46bd-a85f-aa6eca82dbaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 14 08:04:53.957072 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:53.957026    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr" event={"ID":"6d91878b-c89b-4d05-8efa-6836e3ab8872","Type":"ContainerStarted","Data":"0599e4a7074d2cad3cc7fb771cd649c084b3b664c0f88704e0fb8fb4b8e8919a"}
Sep 14 08:04:53.957288 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:53.957088    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr" event={"ID":"6d91878b-c89b-4d05-8efa-6836e3ab8872","Type":"ContainerStarted","Data":"c87e92dee75895b8a24a59390d8b563e0ea17b16ec85940f781bc17c87f61c39"}
Sep 14 08:04:53.980778 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:53.980723    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr" podStartSLOduration=1.980707347 podStartE2EDuration="1.980707347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:04:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:04:53.979921354 +0000 UTC m=+996.436371349" watchObservedRunningTime="2026-09-14 08:04:53.980707347 +0000 UTC m=+996.437157331"
Sep 14 08:04:54.642948 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:54.642899    2562 patch_prober.go:28] interesting pod/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 14 08:04:54.643432 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:54.642978    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s" podUID="fef6e286-c65d-46bd-a85f-aa6eca82dbaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 14 08:04:55.643004 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:55.642962    2562 patch_prober.go:28] interesting pod/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 14 08:04:55.643533 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:55.643033    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s" podUID="fef6e286-c65d-46bd-a85f-aa6eca82dbaa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 14 08:04:56.646488 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:56.646456    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:56.646944 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:56.646784    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:56.647572 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:56.647548    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:04:56.864914 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:56.864879    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:57.418635 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:57.418597    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:57.484979 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:57.484939    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:57.692158 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:57.692121    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:57.793133 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:57.793097    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:57.985004 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:57.984924    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"]
Sep 14 08:04:57.987334 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:57.987316    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:57.990638 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:57.990613    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-af014a32-openshift-default-dockercfg-gxfvv\""
Sep 14 08:04:57.999649 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:57.999623    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"]
Sep 14 08:04:58.015772 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.015733    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:58.119625 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.119593    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-token\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.119822 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.119635    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-data\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.119822 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.119669    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2a003be-8d5e-4d91-aaea-3232ad982854-istiod-ca-cert\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.119822 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.119747    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj8fb\" (UniqueName: \"kubernetes.io/projected/b2a003be-8d5e-4d91-aaea-3232ad982854-kube-api-access-vj8fb\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.119822 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.119788    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-workload-socket\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.120044 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.119880    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-podinfo\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.120044 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.119962    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-credential-socket\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.120129 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.120060    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-workload-certs\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.120185 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.120165    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-envoy\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.220753 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.220698    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-envoy\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.220926 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.220776    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-token\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.220926 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.220802    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-data\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.220926 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.220835    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2a003be-8d5e-4d91-aaea-3232ad982854-istiod-ca-cert\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.221095 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.220983    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj8fb\" (UniqueName: \"kubernetes.io/projected/b2a003be-8d5e-4d91-aaea-3232ad982854-kube-api-access-vj8fb\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.221095 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.221037    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-workload-socket\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.221205 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.221105    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-podinfo\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.221205 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.221186    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-credential-socket\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.221303 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.221215    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-workload-certs\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.221490 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.221438    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-workload-socket\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.221490 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.221485    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-credential-socket\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.221714 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.221667    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2a003be-8d5e-4d91-aaea-3232ad982854-istiod-ca-cert\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.221775 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.221758    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-workload-certs\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.221869 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.221851    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-data\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.223287 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.223267    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-envoy\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.223876 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.223857    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-podinfo\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.235042 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.234982    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-token\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.235345 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.235323    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj8fb\" (UniqueName: \"kubernetes.io/projected/b2a003be-8d5e-4d91-aaea-3232ad982854-kube-api-access-vj8fb\") pod \"e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") " pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.300383 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.300346    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:58.613986 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.613900    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:58.663492 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.663424    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"]
Sep 14 08:04:58.666211 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:04:58.666173    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2a003be_8d5e_4d91_aaea_3232ad982854.slice/crio-f8793ad28c50719ec2f73c4e54e15f8aad53d09a9982d362ee9f9b701b968d6b WatchSource:0}: Error finding container f8793ad28c50719ec2f73c4e54e15f8aad53d09a9982d362ee9f9b701b968d6b: Status 404 returned error can't find the container with id f8793ad28c50719ec2f73c4e54e15f8aad53d09a9982d362ee9f9b701b968d6b
Sep 14 08:04:58.668374 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.668345    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:04:58.668433 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.668424    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:04:58.668471 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.668451    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:04:58.991075 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.991037    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k" event={"ID":"b2a003be-8d5e-4d91-aaea-3232ad982854","Type":"ContainerStarted","Data":"b830278ff59107b99588dbee03f839a1b575e37569aa76ce5a00fd550adb9409"}
Sep 14 08:04:58.991442 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:58.991103    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k" event={"ID":"b2a003be-8d5e-4d91-aaea-3232ad982854","Type":"ContainerStarted","Data":"f8793ad28c50719ec2f73c4e54e15f8aad53d09a9982d362ee9f9b701b968d6b"}
Sep 14 08:04:59.011769 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:59.011702    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k" podStartSLOduration=2.01167929 podStartE2EDuration="2.01167929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:04:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:04:59.010854906 +0000 UTC m=+1001.467304924" watchObservedRunningTime="2026-09-14 08:04:59.01167929 +0000 UTC m=+1001.468129286"
Sep 14 08:04:59.154326 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:59.154287    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:59.300764 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:59.300671    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:59.306054 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:59.306028    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:59.850955 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:59.850921    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:04:59.996077 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:59.996046    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:04:59.997140 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:04:59.997117    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:05:00.134733 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:00.134629    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:01.009772 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:01.009735    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:02.825711 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:02.825667    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:05:02.826167 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:02.825727    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:05:02.838283 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:02.838251    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:05:03.022077 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:03.022048    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:05:03.957197 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:03.957153    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:04.581793 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:04.581755    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:04.680316 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:04.680279    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:04.920514 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:04.920412    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:05.837366 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:05.837333    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:06.634762 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:06.634722    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:07.335093 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:07.335061    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:08.140972 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:08.140929    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:08.240716 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:08.240677    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:08.602800 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:08.602761    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:10.497095 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:10.497058    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:10.598315 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:10.598279    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:10.705559 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:10.705522    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:05:14.591593 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:14.591555    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:29.006798 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:29.006757    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:05:31.315722 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:31.315677    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:32.162659 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:32.162612    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:32.896552 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:32.896521    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:33.835627 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:33.835592    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:34.560702 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:34.560656    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:34.664289 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:34.664257    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:35.300084 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:35.300049    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:35.400300 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:35.400262    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:35.744420 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:35.744377    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:40.216333 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:40.216284    2562 generic.go:356] "Generic (PLEG): container finished" podID="b2a003be-8d5e-4d91-aaea-3232ad982854" containerID="b830278ff59107b99588dbee03f839a1b575e37569aa76ce5a00fd550adb9409" exitCode=0
Sep 14 08:05:40.216869 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:40.216457    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k" event={"ID":"b2a003be-8d5e-4d91-aaea-3232ad982854","Type":"ContainerDied","Data":"b830278ff59107b99588dbee03f839a1b575e37569aa76ce5a00fd550adb9409"}
Sep 14 08:05:40.217029 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:40.217002    2562 scope.go:117] "RemoveContainer" containerID="b830278ff59107b99588dbee03f839a1b575e37569aa76ce5a00fd550adb9409"
Sep 14 08:05:40.218523 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:40.218468    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:05:40.218625 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:40.218582    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:05:40.218688 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:40.218641    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:05:41.223694 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:41.223661    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k" event={"ID":"b2a003be-8d5e-4d91-aaea-3232ad982854","Type":"ContainerStarted","Data":"a38781baf0f9ea7ba0572b9702bb2363ed3058c2977f23a8879ca613ad7176c4"}
Sep 14 08:05:41.300646 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:41.300614    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:05:41.301728 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:41.301700    2562 patch_prober.go:28] interesting pod/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 14 08:05:41.301872 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:41.301757    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k" podUID="b2a003be-8d5e-4d91-aaea-3232ad982854" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 14 08:05:41.697026 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:41.696983    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:41.798465 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:41.798424    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:42.089206 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:42.089123    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-af014a32-bp766"]
Sep 14 08:05:42.093020 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:42.093003    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-af014a32-bp766"
Sep 14 08:05:42.095252 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:42.095225    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:05:42.095385 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:42.095225    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:05:42.099879 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:42.099854    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-af014a32-bp766"]
Sep 14 08:05:42.273465 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:42.273424    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0350c28e-9499-4d91-88d0-bad32a6fd11b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-af014a32-bp766\" (UID: \"0350c28e-9499-4d91-88d0-bad32a6fd11b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-af014a32-bp766"
Sep 14 08:05:42.301655 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:42.301616    2562 patch_prober.go:28] interesting pod/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 14 08:05:42.301849 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:42.301699    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k" podUID="b2a003be-8d5e-4d91-aaea-3232ad982854" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 14 08:05:42.374983 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:42.374879    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0350c28e-9499-4d91-88d0-bad32a6fd11b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-af014a32-bp766\" (UID: \"0350c28e-9499-4d91-88d0-bad32a6fd11b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-af014a32-bp766"
Sep 14 08:05:42.375634 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:42.375609    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0350c28e-9499-4d91-88d0-bad32a6fd11b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-af014a32-bp766\" (UID: \"0350c28e-9499-4d91-88d0-bad32a6fd11b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-af014a32-bp766"
Sep 14 08:05:42.406770 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:42.406736    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-af014a32-bp766"
Sep 14 08:05:42.703120 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:42.703084    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-af014a32-bp766"]
Sep 14 08:05:42.706087 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:05:42.706053    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0350c28e_9499_4d91_88d0_bad32a6fd11b.slice/crio-430fb53435f98d519cfe4ccce41ee7cdd0d7a60ee87dd444883273f8b5a02435 WatchSource:0}: Error finding container 430fb53435f98d519cfe4ccce41ee7cdd0d7a60ee87dd444883273f8b5a02435: Status 404 returned error can't find the container with id 430fb53435f98d519cfe4ccce41ee7cdd0d7a60ee87dd444883273f8b5a02435
Sep 14 08:05:42.809003 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:42.808970    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:42.909350 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:42.909313    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:43.235233 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:43.235139    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-af014a32-bp766" event={"ID":"0350c28e-9499-4d91-88d0-bad32a6fd11b","Type":"ContainerStarted","Data":"d8023db50ee82272a1d23abacf735ea6570b9d1ceb1fbaeaa247db229fb3d072"}
Sep 14 08:05:43.235233 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:43.235200    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-af014a32-bp766" event={"ID":"0350c28e-9499-4d91-88d0-bad32a6fd11b","Type":"ContainerStarted","Data":"430fb53435f98d519cfe4ccce41ee7cdd0d7a60ee87dd444883273f8b5a02435"}
Sep 14 08:05:43.301184 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:43.301146    2562 patch_prober.go:28] interesting pod/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 14 08:05:43.301582 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:43.301223    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k" podUID="b2a003be-8d5e-4d91-aaea-3232ad982854" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 14 08:05:43.746401 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:43.746367    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:44.244477 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:44.244434    2562 generic.go:356] "Generic (PLEG): container finished" podID="0350c28e-9499-4d91-88d0-bad32a6fd11b" containerID="d8023db50ee82272a1d23abacf735ea6570b9d1ceb1fbaeaa247db229fb3d072" exitCode=0
Sep 14 08:05:44.244711 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:44.244560    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-af014a32-bp766" event={"ID":"0350c28e-9499-4d91-88d0-bad32a6fd11b","Type":"ContainerDied","Data":"d8023db50ee82272a1d23abacf735ea6570b9d1ceb1fbaeaa247db229fb3d072"}
Sep 14 08:05:44.301446 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:44.301414    2562 patch_prober.go:28] interesting pod/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 14 08:05:44.301814 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:44.301484    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k" podUID="b2a003be-8d5e-4d91-aaea-3232ad982854" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 14 08:05:44.499008 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:44.498926    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:44.996630 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:44.996584    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:05:45.189376 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:45.189340    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:45.301066 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:45.300914    2562 patch_prober.go:28] interesting pod/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 14 08:05:45.301066 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:45.300975    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k" podUID="b2a003be-8d5e-4d91-aaea-3232ad982854" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 14 08:05:45.419161 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:45.419135    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-af014a32-bp766"
Sep 14 08:05:45.605914 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:45.605817    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0350c28e-9499-4d91-88d0-bad32a6fd11b-maas-api-service-ca\") pod \"0350c28e-9499-4d91-88d0-bad32a6fd11b\" (UID: \"0350c28e-9499-4d91-88d0-bad32a6fd11b\") "
Sep 14 08:05:45.606198 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:45.606172    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0350c28e-9499-4d91-88d0-bad32a6fd11b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0350c28e-9499-4d91-88d0-bad32a6fd11b" (UID: "0350c28e-9499-4d91-88d0-bad32a6fd11b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:05:45.706989 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:45.706941    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0350c28e-9499-4d91-88d0-bad32a6fd11b-maas-api-service-ca\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:05:45.950297 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:45.950253    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:46.260887 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:46.260788    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-af014a32-bp766"
Sep 14 08:05:46.260887 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:46.260790    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-af014a32-bp766" event={"ID":"0350c28e-9499-4d91-88d0-bad32a6fd11b","Type":"ContainerDied","Data":"430fb53435f98d519cfe4ccce41ee7cdd0d7a60ee87dd444883273f8b5a02435"}
Sep 14 08:05:46.261135 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:46.260915    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="430fb53435f98d519cfe4ccce41ee7cdd0d7a60ee87dd444883273f8b5a02435"
Sep 14 08:05:46.324842 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:46.324805    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:05:46.328384 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:46.328357    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:05:46.659450 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:46.659352    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:52.725538 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:52.725493    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:52.794918 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:52.794882    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:52.860718 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:52.860682    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:52.961879 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:52.961846    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-af014a32-bp766"]
Sep 14 08:05:52.964773 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:52.964739    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:52.968105 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:52.968074    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-af014a32-bp766"]
Sep 14 08:05:53.040679 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:53.040581    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:53.255235 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:53.255201    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:53.315684 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:53.315584    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:54.057440 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:54.057403    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:54.113862 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:54.113821    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0350c28e-9499-4d91-88d0-bad32a6fd11b" path="/var/lib/kubelet/pods/0350c28e-9499-4d91-88d0-bad32a6fd11b/volumes"
Sep 14 08:05:54.838604 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:54.838572    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:55.742150 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:55.742115    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:56.046963 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:56.046868    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:56.148091 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:56.148021    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:56.482459 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:56.482425    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:59.096729 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:59.096686    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:59.275608 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:59.275569    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:59.301994 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:59.301960    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:05:59.379704 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:59.379600    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:59.497734 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:59.497697    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"]
Sep 14 08:05:59.497963 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:59.497926    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k" podUID="b2a003be-8d5e-4d91-aaea-3232ad982854" containerName="istio-proxy" containerID="cri-o://a38781baf0f9ea7ba0572b9702bb2363ed3058c2977f23a8879ca613ad7176c4" gracePeriod=30
Sep 14 08:05:59.511404 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:59.511375    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:59.864877 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:59.864846    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:05:59.974843 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:05:59.974806    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:00.107038 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:00.107005    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:01.328187 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:01.328146    2562 patch_prober.go:28] interesting pod/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 14 08:06:01.328603 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:01.328226    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k" podUID="b2a003be-8d5e-4d91-aaea-3232ad982854" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 08:06:01.980050 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:01.980011    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:04.751605 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.751576    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:06:04.786213 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.786130    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-data\") pod \"b2a003be-8d5e-4d91-aaea-3232ad982854\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") "
Sep 14 08:06:04.786386 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.786214    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-workload-certs\") pod \"b2a003be-8d5e-4d91-aaea-3232ad982854\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") "
Sep 14 08:06:04.786386 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.786251    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-workload-socket\") pod \"b2a003be-8d5e-4d91-aaea-3232ad982854\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") "
Sep 14 08:06:04.786386 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.786280    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-envoy\") pod \"b2a003be-8d5e-4d91-aaea-3232ad982854\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") "
Sep 14 08:06:04.786386 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.786311    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-podinfo\") pod \"b2a003be-8d5e-4d91-aaea-3232ad982854\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") "
Sep 14 08:06:04.786386 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.786341    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-credential-socket\") pod \"b2a003be-8d5e-4d91-aaea-3232ad982854\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") "
Sep 14 08:06:04.786705 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.786396    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2a003be-8d5e-4d91-aaea-3232ad982854-istiod-ca-cert\") pod \"b2a003be-8d5e-4d91-aaea-3232ad982854\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") "
Sep 14 08:06:04.786705 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.786431    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vj8fb\" (UniqueName: \"kubernetes.io/projected/b2a003be-8d5e-4d91-aaea-3232ad982854-kube-api-access-vj8fb\") pod \"b2a003be-8d5e-4d91-aaea-3232ad982854\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") "
Sep 14 08:06:04.786705 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.786434    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-data" (OuterVolumeSpecName: "istio-data") pod "b2a003be-8d5e-4d91-aaea-3232ad982854" (UID: "b2a003be-8d5e-4d91-aaea-3232ad982854"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:06:04.786705 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.786461    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-token\") pod \"b2a003be-8d5e-4d91-aaea-3232ad982854\" (UID: \"b2a003be-8d5e-4d91-aaea-3232ad982854\") "
Sep 14 08:06:04.786705 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.786466    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b2a003be-8d5e-4d91-aaea-3232ad982854" (UID: "b2a003be-8d5e-4d91-aaea-3232ad982854"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:06:04.786705 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.786590    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b2a003be-8d5e-4d91-aaea-3232ad982854" (UID: "b2a003be-8d5e-4d91-aaea-3232ad982854"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:06:04.787032 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.786787    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2a003be-8d5e-4d91-aaea-3232ad982854-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b2a003be-8d5e-4d91-aaea-3232ad982854" (UID: "b2a003be-8d5e-4d91-aaea-3232ad982854"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:06:04.787032 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.786958    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b2a003be-8d5e-4d91-aaea-3232ad982854" (UID: "b2a003be-8d5e-4d91-aaea-3232ad982854"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:06:04.787032 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.786994    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-workload-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:06:04.787032 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.787021    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-workload-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:06:04.787251 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.787037    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2a003be-8d5e-4d91-aaea-3232ad982854-istiod-ca-cert\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:06:04.787251 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.787050    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-data\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:06:04.788754 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.788720    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b2a003be-8d5e-4d91-aaea-3232ad982854" (UID: "b2a003be-8d5e-4d91-aaea-3232ad982854"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:06:04.788754 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.788740    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b2a003be-8d5e-4d91-aaea-3232ad982854" (UID: "b2a003be-8d5e-4d91-aaea-3232ad982854"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:06:04.789000 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.788981    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-token" (OuterVolumeSpecName: "istio-token") pod "b2a003be-8d5e-4d91-aaea-3232ad982854" (UID: "b2a003be-8d5e-4d91-aaea-3232ad982854"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:06:04.789042 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.788981    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2a003be-8d5e-4d91-aaea-3232ad982854-kube-api-access-vj8fb" (OuterVolumeSpecName: "kube-api-access-vj8fb") pod "b2a003be-8d5e-4d91-aaea-3232ad982854" (UID: "b2a003be-8d5e-4d91-aaea-3232ad982854"). InnerVolumeSpecName "kube-api-access-vj8fb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:06:04.888014 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.887971    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-credential-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:06:04.888014 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.888007    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vj8fb\" (UniqueName: \"kubernetes.io/projected/b2a003be-8d5e-4d91-aaea-3232ad982854-kube-api-access-vj8fb\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:06:04.888014 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.888018    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-token\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:06:04.888249 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.888027    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-envoy\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:06:04.888249 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:04.888036    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2a003be-8d5e-4d91-aaea-3232ad982854-istio-podinfo\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:06:05.362395 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:05.362358    2562 generic.go:356] "Generic (PLEG): container finished" podID="b2a003be-8d5e-4d91-aaea-3232ad982854" containerID="a38781baf0f9ea7ba0572b9702bb2363ed3058c2977f23a8879ca613ad7176c4" exitCode=0
Sep 14 08:06:05.362628 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:05.362411    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"
Sep 14 08:06:05.362628 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:05.362439    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k" event={"ID":"b2a003be-8d5e-4d91-aaea-3232ad982854","Type":"ContainerDied","Data":"a38781baf0f9ea7ba0572b9702bb2363ed3058c2977f23a8879ca613ad7176c4"}
Sep 14 08:06:05.362628 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:05.362494    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k" event={"ID":"b2a003be-8d5e-4d91-aaea-3232ad982854","Type":"ContainerDied","Data":"f8793ad28c50719ec2f73c4e54e15f8aad53d09a9982d362ee9f9b701b968d6b"}
Sep 14 08:06:05.362628 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:05.362530    2562 scope.go:117] "RemoveContainer" containerID="a38781baf0f9ea7ba0572b9702bb2363ed3058c2977f23a8879ca613ad7176c4"
Sep 14 08:06:05.371992 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:05.371975    2562 scope.go:117] "RemoveContainer" containerID="b830278ff59107b99588dbee03f839a1b575e37569aa76ce5a00fd550adb9409"
Sep 14 08:06:05.379955 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:05.379936    2562 scope.go:117] "RemoveContainer" containerID="a38781baf0f9ea7ba0572b9702bb2363ed3058c2977f23a8879ca613ad7176c4"
Sep 14 08:06:05.380239 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:06:05.380202    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a38781baf0f9ea7ba0572b9702bb2363ed3058c2977f23a8879ca613ad7176c4\": container with ID starting with a38781baf0f9ea7ba0572b9702bb2363ed3058c2977f23a8879ca613ad7176c4 not found: ID does not exist" containerID="a38781baf0f9ea7ba0572b9702bb2363ed3058c2977f23a8879ca613ad7176c4"
Sep 14 08:06:05.380299 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:05.380247    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a38781baf0f9ea7ba0572b9702bb2363ed3058c2977f23a8879ca613ad7176c4"} err="failed to get container status \"a38781baf0f9ea7ba0572b9702bb2363ed3058c2977f23a8879ca613ad7176c4\": rpc error: code = NotFound desc = could not find container \"a38781baf0f9ea7ba0572b9702bb2363ed3058c2977f23a8879ca613ad7176c4\": container with ID starting with a38781baf0f9ea7ba0572b9702bb2363ed3058c2977f23a8879ca613ad7176c4 not found: ID does not exist"
Sep 14 08:06:05.380299 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:05.380265    2562 scope.go:117] "RemoveContainer" containerID="b830278ff59107b99588dbee03f839a1b575e37569aa76ce5a00fd550adb9409"
Sep 14 08:06:05.380534 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:06:05.380519    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b830278ff59107b99588dbee03f839a1b575e37569aa76ce5a00fd550adb9409\": container with ID starting with b830278ff59107b99588dbee03f839a1b575e37569aa76ce5a00fd550adb9409 not found: ID does not exist" containerID="b830278ff59107b99588dbee03f839a1b575e37569aa76ce5a00fd550adb9409"
Sep 14 08:06:05.380606 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:05.380537    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b830278ff59107b99588dbee03f839a1b575e37569aa76ce5a00fd550adb9409"} err="failed to get container status \"b830278ff59107b99588dbee03f839a1b575e37569aa76ce5a00fd550adb9409\": rpc error: code = NotFound desc = could not find container \"b830278ff59107b99588dbee03f839a1b575e37569aa76ce5a00fd550adb9409\": container with ID starting with b830278ff59107b99588dbee03f839a1b575e37569aa76ce5a00fd550adb9409 not found: ID does not exist"
Sep 14 08:06:05.384167 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:05.384145    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"]
Sep 14 08:06:05.387355 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:05.387334    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-af014a32-openshift-default-6b44db4448-9wn5k"]
Sep 14 08:06:05.769255 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:05.769218    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:05.870071 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:05.870038    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:05.969622 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:05.969588    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:06.112599 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:06.112522    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2a003be-8d5e-4d91-aaea-3232ad982854" path="/var/lib/kubelet/pods/b2a003be-8d5e-4d91-aaea-3232ad982854/volumes"
Sep 14 08:06:08.349634 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:08.349591    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:11.937653 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:11.937572    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:12.040103 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:12.040065    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:12.937471 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:12.937435    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:13.219869 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.219782    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"]
Sep 14 08:06:13.220846 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.220824    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2a003be-8d5e-4d91-aaea-3232ad982854" containerName="istio-proxy"
Sep 14 08:06:13.220846 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.220848    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2a003be-8d5e-4d91-aaea-3232ad982854" containerName="istio-proxy"
Sep 14 08:06:13.220967 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.220858    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2a003be-8d5e-4d91-aaea-3232ad982854" containerName="istio-proxy"
Sep 14 08:06:13.220967 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.220864    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2a003be-8d5e-4d91-aaea-3232ad982854" containerName="istio-proxy"
Sep 14 08:06:13.220967 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.220883    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0350c28e-9499-4d91-88d0-bad32a6fd11b" containerName="revoke-keys"
Sep 14 08:06:13.220967 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.220889    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="0350c28e-9499-4d91-88d0-bad32a6fd11b" containerName="revoke-keys"
Sep 14 08:06:13.220967 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.220953    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2a003be-8d5e-4d91-aaea-3232ad982854" containerName="istio-proxy"
Sep 14 08:06:13.220967 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.220961    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2a003be-8d5e-4d91-aaea-3232ad982854" containerName="istio-proxy"
Sep 14 08:06:13.220967 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.220969    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="0350c28e-9499-4d91-88d0-bad32a6fd11b" containerName="revoke-keys"
Sep 14 08:06:13.225178 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.225156    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.227250 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.227228    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-b990f4-openshift-default-dockercfg-w8997\""
Sep 14 08:06:13.234529 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.234493    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"]
Sep 14 08:06:13.248918 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.248887    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:13.348971 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.348935    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:13.375263 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.375227    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-credential-socket\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.375423 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.375289    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-workload-socket\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.375423 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.375336    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-istio-data\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.375423 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.375364    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934c8412-b086-4b93-87e9-69b2df9826e2-istiod-ca-cert\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.375423 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.375409    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-workload-certs\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.375664 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.375441    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934c8412-b086-4b93-87e9-69b2df9826e2-istio-podinfo\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.375664 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.375610    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-istio-envoy\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.375768 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.375674    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/934c8412-b086-4b93-87e9-69b2df9826e2-istio-token\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.375768 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.375710    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdltj\" (UniqueName: \"kubernetes.io/projected/934c8412-b086-4b93-87e9-69b2df9826e2-kube-api-access-bdltj\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.476935 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.476832    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-istio-envoy\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.476935 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.476885    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/934c8412-b086-4b93-87e9-69b2df9826e2-istio-token\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.476935 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.476907    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdltj\" (UniqueName: \"kubernetes.io/projected/934c8412-b086-4b93-87e9-69b2df9826e2-kube-api-access-bdltj\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.477235 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.476956    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-credential-socket\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.477235 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.476982    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-workload-socket\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.477235 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.477013    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-istio-data\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.477235 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.477037    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934c8412-b086-4b93-87e9-69b2df9826e2-istiod-ca-cert\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.477235 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.477062    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-workload-certs\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.477235 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.477094    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934c8412-b086-4b93-87e9-69b2df9826e2-istio-podinfo\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.477573 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.477406    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-istio-data\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.477573 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.477435    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-credential-socket\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.477670 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.477567    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-workload-socket\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.477670 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.477610    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-workload-certs\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.477869 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.477851    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934c8412-b086-4b93-87e9-69b2df9826e2-istiod-ca-cert\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.479220 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.479199    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-istio-envoy\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.479460 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.479432    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934c8412-b086-4b93-87e9-69b2df9826e2-istio-podinfo\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.485388 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.485361    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/934c8412-b086-4b93-87e9-69b2df9826e2-istio-token\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.485495 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.485468    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdltj\" (UniqueName: \"kubernetes.io/projected/934c8412-b086-4b93-87e9-69b2df9826e2-kube-api-access-bdltj\") pod \"e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") " pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.537408 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.537370    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:13.815470 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.815362    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"]
Sep 14 08:06:13.818535 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:06:13.818487    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod934c8412_b086_4b93_87e9_69b2df9826e2.slice/crio-522544a0a36435bc56c463962b4e39b09ac012dba934d68b96b5033b4e57a453 WatchSource:0}: Error finding container 522544a0a36435bc56c463962b4e39b09ac012dba934d68b96b5033b4e57a453: Status 404 returned error can't find the container with id 522544a0a36435bc56c463962b4e39b09ac012dba934d68b96b5033b4e57a453
Sep 14 08:06:13.820846 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.820812    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:06:13.820942 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.820896    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:06:13.821006 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:13.820939    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:06:14.413592 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:14.413554    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp" event={"ID":"934c8412-b086-4b93-87e9-69b2df9826e2","Type":"ContainerStarted","Data":"52850a5bbb296da0e0c5bf4e503eb70afa75e7f5dde55e9a4803290d5fa079c0"}
Sep 14 08:06:14.413993 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:14.413608    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp" event={"ID":"934c8412-b086-4b93-87e9-69b2df9826e2","Type":"ContainerStarted","Data":"522544a0a36435bc56c463962b4e39b09ac012dba934d68b96b5033b4e57a453"}
Sep 14 08:06:14.435367 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:14.435305    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp" podStartSLOduration=1.435286766 podStartE2EDuration="1.435286766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:06:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:06:14.434238736 +0000 UTC m=+1076.890688746" watchObservedRunningTime="2026-09-14 08:06:14.435286766 +0000 UTC m=+1076.891736758"
Sep 14 08:06:14.537833 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:14.537798    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:14.543387 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:14.543356    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:15.419231 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:15.419194    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:15.420151 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:15.420131    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:15.702476 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:15.702445    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:16.351804 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:16.351765    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:16.451511 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:16.451464    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:16.546624 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:16.546592    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:17.335739 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:17.335697    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:18.164424 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:18.164387    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:19.021553 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:19.021521    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:19.732992 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:19.732956    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:19.833794 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:19.833762    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:20.026543 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:20.026442    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:20.210302 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:20.210260    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:20.310059 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:20.309968    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:29.466246 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:29.466201    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:30.401598 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:30.401562    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:06:38.788659 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.788620    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"]
Sep 14 08:06:38.793973 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.793943    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:38.795979 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.795948    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-73bd4a\"/\"default-dockercfg-mq8pd\""
Sep 14 08:06:38.796129 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.796006    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-73bd4a\"/\"kube-root-ca.crt\""
Sep 14 08:06:38.796129 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.796076    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-73bd4a\"/\"openshift-service-ca.crt\""
Sep 14 08:06:38.796546 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.796525    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-73bd4a\"/\"explicit-model-73bd4a-kserve-self-signed-certs\""
Sep 14 08:06:38.800458 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.800428    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"]
Sep 14 08:06:38.815250 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.815208    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:38.839382 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.839345    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-model-cache\") pod \"explicit-model-73bd4a-kserve-596f74bbf9-xxhnv\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") " pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:38.839582 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.839431    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-home\") pod \"explicit-model-73bd4a-kserve-596f74bbf9-xxhnv\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") " pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:38.839582 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.839464    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-tmp-dir\") pod \"explicit-model-73bd4a-kserve-596f74bbf9-xxhnv\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") " pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:38.839582 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.839487    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq48j\" (UniqueName: \"kubernetes.io/projected/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-kube-api-access-lq48j\") pod \"explicit-model-73bd4a-kserve-596f74bbf9-xxhnv\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") " pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:38.839582 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.839549    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-dshm\") pod \"explicit-model-73bd4a-kserve-596f74bbf9-xxhnv\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") " pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:38.839730 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.839611    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-tls-certs\") pod \"explicit-model-73bd4a-kserve-596f74bbf9-xxhnv\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") " pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:38.940429 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.940383    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-model-cache\") pod \"explicit-model-73bd4a-kserve-596f74bbf9-xxhnv\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") " pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:38.940651 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.940455    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-home\") pod \"explicit-model-73bd4a-kserve-596f74bbf9-xxhnv\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") " pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:38.940651 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.940470    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-tmp-dir\") pod \"explicit-model-73bd4a-kserve-596f74bbf9-xxhnv\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") " pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:38.940651 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.940489    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq48j\" (UniqueName: \"kubernetes.io/projected/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-kube-api-access-lq48j\") pod \"explicit-model-73bd4a-kserve-596f74bbf9-xxhnv\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") " pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:38.940651 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.940521    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-dshm\") pod \"explicit-model-73bd4a-kserve-596f74bbf9-xxhnv\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") " pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:38.940917 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.940883    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-tls-certs\") pod \"explicit-model-73bd4a-kserve-596f74bbf9-xxhnv\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") " pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:38.941061 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.941029    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-home\") pod \"explicit-model-73bd4a-kserve-596f74bbf9-xxhnv\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") " pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:38.941184 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.941079    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-tmp-dir\") pod \"explicit-model-73bd4a-kserve-596f74bbf9-xxhnv\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") " pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:38.941184 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.940913    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-model-cache\") pod \"explicit-model-73bd4a-kserve-596f74bbf9-xxhnv\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") " pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:38.943225 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.943199    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-dshm\") pod \"explicit-model-73bd4a-kserve-596f74bbf9-xxhnv\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") " pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:38.943339 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.943270    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-tls-certs\") pod \"explicit-model-73bd4a-kserve-596f74bbf9-xxhnv\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") " pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:38.951062 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:38.951038    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq48j\" (UniqueName: \"kubernetes.io/projected/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-kube-api-access-lq48j\") pod \"explicit-model-73bd4a-kserve-596f74bbf9-xxhnv\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") " pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:39.107309 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:39.107173    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:39.389360 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:39.389331    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"]
Sep 14 08:06:39.390620 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:06:39.390594    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaba992c0_2f6e_4b97_b69e_7194f3e3be7c.slice/crio-a4a6cc30c396069ced37534ce9ee170e72b796a03ae65041e19065f060e91f2d WatchSource:0}: Error finding container a4a6cc30c396069ced37534ce9ee170e72b796a03ae65041e19065f060e91f2d: Status 404 returned error can't find the container with id a4a6cc30c396069ced37534ce9ee170e72b796a03ae65041e19065f060e91f2d
Sep 14 08:06:39.551677 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:39.551635    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv" event={"ID":"aba992c0-2f6e-4b97-b69e-7194f3e3be7c","Type":"ContainerStarted","Data":"768f17a406300abad23995365588e1f8bf206d869ea3792b22ee378d07c2f48d"}
Sep 14 08:06:39.551848 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:39.551688    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv" event={"ID":"aba992c0-2f6e-4b97-b69e-7194f3e3be7c","Type":"ContainerStarted","Data":"a4a6cc30c396069ced37534ce9ee170e72b796a03ae65041e19065f060e91f2d"}
Sep 14 08:06:39.567920 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:39.567864    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv" podStartSLOduration=1.567849099 podStartE2EDuration="1.567849099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:06:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:06:39.566613308 +0000 UTC m=+1102.023063301" watchObservedRunningTime="2026-09-14 08:06:39.567849099 +0000 UTC m=+1102.024299092"
Sep 14 08:06:40.634839 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:40.634806    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:41.564094 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:41.564058    2562 generic.go:356] "Generic (PLEG): container finished" podID="934c8412-b086-4b93-87e9-69b2df9826e2" containerID="52850a5bbb296da0e0c5bf4e503eb70afa75e7f5dde55e9a4803290d5fa079c0" exitCode=0
Sep 14 08:06:41.564310 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:41.564134    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp" event={"ID":"934c8412-b086-4b93-87e9-69b2df9826e2","Type":"ContainerDied","Data":"52850a5bbb296da0e0c5bf4e503eb70afa75e7f5dde55e9a4803290d5fa079c0"}
Sep 14 08:06:41.564596 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:41.564582    2562 scope.go:117] "RemoveContainer" containerID="52850a5bbb296da0e0c5bf4e503eb70afa75e7f5dde55e9a4803290d5fa079c0"
Sep 14 08:06:41.565721 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:41.565692    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:06:41.565787 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:41.565765    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:06:41.565823 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:41.565798    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:06:42.058453 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:42.058413    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:42.571845 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:42.571806    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp" event={"ID":"934c8412-b086-4b93-87e9-69b2df9826e2","Type":"ContainerStarted","Data":"04e3b4c076c3dfdd770933c49b6dbeb17669c1eab8317a1b2b53d66b903a43ec"}
Sep 14 08:06:42.990127 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:42.990094    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:43.538570 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:43.538531    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:43.539840 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:43.539800    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 14 08:06:43.539992 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:43.539885    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp" podUID="934c8412-b086-4b93-87e9-69b2df9826e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 14 08:06:43.671087 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:43.671044    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:44.426342 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:44.426305    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:44.538780 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:44.538734    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 14 08:06:44.539265 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:44.538807    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp" podUID="934c8412-b086-4b93-87e9-69b2df9826e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 14 08:06:45.354058 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:45.354020    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:45.419925 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:45.419876    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:45.538763 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:45.538719    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 14 08:06:45.538972 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:45.538795    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp" podUID="934c8412-b086-4b93-87e9-69b2df9826e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 14 08:06:46.038814 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:46.038778    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:46.317384 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:46.317292    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:06:46.446277 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:46.446236    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:46.538284 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:46.538241    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 14 08:06:46.538449 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:46.538308    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp" podUID="934c8412-b086-4b93-87e9-69b2df9826e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 14 08:06:46.784711 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:46.784668    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:47.061758 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.061660    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"]
Sep 14 08:06:47.074137 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.074108    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.076643 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.076609    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-afcf8a-openshift-default-dockercfg-47mhn\""
Sep 14 08:06:47.083477 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.083439    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"]
Sep 14 08:06:47.088302 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.088271    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:47.131372 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.131331    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/896de534-987b-452c-a3a3-960f67ed3cc8-istio-token\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.131372 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.131376    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-workload-socket\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.131644 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.131406    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-workload-certs\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.131644 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.131539    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/896de534-987b-452c-a3a3-960f67ed3cc8-istio-podinfo\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.131644 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.131565    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-istio-data\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.131644 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.131590    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5546b\" (UniqueName: \"kubernetes.io/projected/896de534-987b-452c-a3a3-960f67ed3cc8-kube-api-access-5546b\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.131781 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.131663    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-credential-socket\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.131781 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.131725    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/896de534-987b-452c-a3a3-960f67ed3cc8-istiod-ca-cert\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.131855 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.131793    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-istio-envoy\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.232893 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.232847    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-istio-envoy\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.233111 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.232912    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/896de534-987b-452c-a3a3-960f67ed3cc8-istio-token\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.233111 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.232943    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-workload-socket\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.233111 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.232979    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-workload-certs\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.233111 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.233050    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/896de534-987b-452c-a3a3-960f67ed3cc8-istio-podinfo\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.233111 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.233076    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-istio-data\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.233391 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.233110    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5546b\" (UniqueName: \"kubernetes.io/projected/896de534-987b-452c-a3a3-960f67ed3cc8-kube-api-access-5546b\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.233391 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.233149    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-credential-socket\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.233391 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.233201    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/896de534-987b-452c-a3a3-960f67ed3cc8-istiod-ca-cert\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.233657 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.233626    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-istio-data\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.233886 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.233865    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-workload-socket\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.234090 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.234059    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-credential-socket\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.234196 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.234150    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-workload-certs\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.234295 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.234273    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/896de534-987b-452c-a3a3-960f67ed3cc8-istiod-ca-cert\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.235168 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.235149    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-istio-envoy\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.236033 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.236010    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/896de534-987b-452c-a3a3-960f67ed3cc8-istio-podinfo\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.244810 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.244755    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/896de534-987b-452c-a3a3-960f67ed3cc8-istio-token\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.249227 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.249172    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5546b\" (UniqueName: \"kubernetes.io/projected/896de534-987b-452c-a3a3-960f67ed3cc8-kube-api-access-5546b\") pod \"e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") " pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.389467 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.389382    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:47.538338 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.538248    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 14 08:06:47.538565 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.538347    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp" podUID="934c8412-b086-4b93-87e9-69b2df9826e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 14 08:06:47.703110 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.703074    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"]
Sep 14 08:06:47.709528 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:06:47.709462    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod896de534_987b_452c_a3a3_960f67ed3cc8.slice/crio-ac0b7908e8d8dc052fe238bddf1b19b1b9323f71219dd818526aea3b383ba34b WatchSource:0}: Error finding container ac0b7908e8d8dc052fe238bddf1b19b1b9323f71219dd818526aea3b383ba34b: Status 404 returned error can't find the container with id ac0b7908e8d8dc052fe238bddf1b19b1b9323f71219dd818526aea3b383ba34b
Sep 14 08:06:47.711803 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.711759    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:06:47.711927 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.711855    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:06:47.711927 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.711898    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:06:47.769444 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:47.769408    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:48.541965 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:48.541922    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:48.542842 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:48.542824    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:06:48.615916 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:48.615882    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878" event={"ID":"896de534-987b-452c-a3a3-960f67ed3cc8","Type":"ContainerStarted","Data":"331fea5c716ada12f03eaee4b5f0fb7a548b2ad03659ae20b54f2670df9e70db"}
Sep 14 08:06:48.616085 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:48.615939    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878" event={"ID":"896de534-987b-452c-a3a3-960f67ed3cc8","Type":"ContainerStarted","Data":"ac0b7908e8d8dc052fe238bddf1b19b1b9323f71219dd818526aea3b383ba34b"}
Sep 14 08:06:48.634427 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:48.634373    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878" podStartSLOduration=1.634355995 podStartE2EDuration="1.634355995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:06:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:06:48.631486815 +0000 UTC m=+1111.087936809" watchObservedRunningTime="2026-09-14 08:06:48.634355995 +0000 UTC m=+1111.090805987"
Sep 14 08:06:49.107512 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:49.107476    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:49.107687 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:49.107545    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:49.119953 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:49.119916    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:49.390125 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:49.390025    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:49.395287 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:49.395257    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:49.621118 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:49.621079    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:49.621963 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:49.621939    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:06:49.632399 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:49.632373    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:06:49.938453 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:49.938409    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:50.542109 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:50.542071    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:50.641817 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:50.641780    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:50.751635 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:50.751600    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:51.613150 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:51.613114    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:52.523986 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:52.523947    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:53.232997 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:53.232961    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:53.927646 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:53.927610    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:54.758418 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:54.758373    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:54.857394 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:54.857361    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:55.191813 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:55.191784    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:55.292883 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:55.292849    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:06:55.368093 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:06:55.368062    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:00.294910 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:00.294868    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:04.216938 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:04.216895    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:05.692153 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:05.692116    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"]
Sep 14 08:07:05.721839 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:05.721807    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:05.810896 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:05.810863    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-73bd4a-4jb69"]
Sep 14 08:07:05.814538 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:05.814516    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-73bd4a-4jb69"
Sep 14 08:07:05.816652 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:05.816633    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:07:05.816739 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:05.816634    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:07:05.821917 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:05.821853    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-73bd4a-4jb69"]
Sep 14 08:07:05.830807 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:05.830776    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72df8032-0167-4db0-b21a-7aebdba43d98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-73bd4a-4jb69\" (UID: \"72df8032-0167-4db0-b21a-7aebdba43d98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-73bd4a-4jb69"
Sep 14 08:07:05.931810 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:05.931765    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72df8032-0167-4db0-b21a-7aebdba43d98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-73bd4a-4jb69\" (UID: \"72df8032-0167-4db0-b21a-7aebdba43d98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-73bd4a-4jb69"
Sep 14 08:07:05.932411 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:05.932390    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72df8032-0167-4db0-b21a-7aebdba43d98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-73bd4a-4jb69\" (UID: \"72df8032-0167-4db0-b21a-7aebdba43d98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-73bd4a-4jb69"
Sep 14 08:07:06.128050 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:06.127959    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-73bd4a-4jb69"
Sep 14 08:07:06.141495 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:06.141469    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:06.398757 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:06.398728    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-73bd4a-4jb69"]
Sep 14 08:07:06.400691 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:07:06.400659    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72df8032_0167_4db0_b21a_7aebdba43d98.slice/crio-87dbbe248a261e764d360ee74884e3b6713c2d89b833d88e3e6fa067cd4b581d WatchSource:0}: Error finding container 87dbbe248a261e764d360ee74884e3b6713c2d89b833d88e3e6fa067cd4b581d: Status 404 returned error can't find the container with id 87dbbe248a261e764d360ee74884e3b6713c2d89b833d88e3e6fa067cd4b581d
Sep 14 08:07:06.711898 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:06.711859    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-73bd4a-4jb69" event={"ID":"72df8032-0167-4db0-b21a-7aebdba43d98","Type":"ContainerStarted","Data":"53921550fb615ce9c8d6e2d8a50ca96d987d213a699b1c8ef845c5b4bcf20d93"}
Sep 14 08:07:06.712272 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:06.711917    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-73bd4a-4jb69" event={"ID":"72df8032-0167-4db0-b21a-7aebdba43d98","Type":"ContainerStarted","Data":"87dbbe248a261e764d360ee74884e3b6713c2d89b833d88e3e6fa067cd4b581d"}
Sep 14 08:07:06.725934 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:06.725881    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-73bd4a-4jb69" podStartSLOduration=1.725864948 podStartE2EDuration="1.725864948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:06.725600675 +0000 UTC m=+1129.182050668" watchObservedRunningTime="2026-09-14 08:07:06.725864948 +0000 UTC m=+1129.182314940"
Sep 14 08:07:06.888361 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:06.888322    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"]
Sep 14 08:07:06.904813 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:06.904713    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:07.003810 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:07.003772    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:07.015352 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:07.015315    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:07:07.719062 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:07.719022    2562 generic.go:356] "Generic (PLEG): container finished" podID="72df8032-0167-4db0-b21a-7aebdba43d98" containerID="53921550fb615ce9c8d6e2d8a50ca96d987d213a699b1c8ef845c5b4bcf20d93" exitCode=0
Sep 14 08:07:07.719062 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:07.719066    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-73bd4a-4jb69" event={"ID":"72df8032-0167-4db0-b21a-7aebdba43d98","Type":"ContainerDied","Data":"53921550fb615ce9c8d6e2d8a50ca96d987d213a699b1c8ef845c5b4bcf20d93"}
Sep 14 08:07:07.859183 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:07.859137    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:08.598762 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:08.598726    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:08.861405 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:08.861341    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-73bd4a-4jb69"
Sep 14 08:07:08.960146 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:08.960108    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72df8032-0167-4db0-b21a-7aebdba43d98-maas-api-service-ca\") pod \"72df8032-0167-4db0-b21a-7aebdba43d98\" (UID: \"72df8032-0167-4db0-b21a-7aebdba43d98\") "
Sep 14 08:07:08.960555 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:08.960496    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72df8032-0167-4db0-b21a-7aebdba43d98-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "72df8032-0167-4db0-b21a-7aebdba43d98" (UID: "72df8032-0167-4db0-b21a-7aebdba43d98"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:09.061527 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:09.061455    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/72df8032-0167-4db0-b21a-7aebdba43d98-maas-api-service-ca\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:09.522282 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:09.522249    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:09.731678 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:09.731646    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-73bd4a-4jb69"
Sep 14 08:07:09.731846 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:09.731646    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-73bd4a-4jb69" event={"ID":"72df8032-0167-4db0-b21a-7aebdba43d98","Type":"ContainerDied","Data":"87dbbe248a261e764d360ee74884e3b6713c2d89b833d88e3e6fa067cd4b581d"}
Sep 14 08:07:09.731846 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:09.731786    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="87dbbe248a261e764d360ee74884e3b6713c2d89b833d88e3e6fa067cd4b581d"
Sep 14 08:07:10.260280 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:10.260246    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:10.489428 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:10.489391    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:11.144434 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:11.144400    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:11.863592 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:11.863538    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:12.648247 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:12.648208    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:13.364491 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:13.364445    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:14.059369 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:14.059322    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:14.797345 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:14.797306    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:15.998063 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:15.998021    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:16.730697 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:16.730665    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:16.798947 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:16.798910    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:16.856257 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:16.856227    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:17.013340 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:17.013245    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:17.184100 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:17.184071    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:17.188963 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:17.188929    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-73bd4a-4jb69"]
Sep 14 08:07:17.192541 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:17.192511    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-73bd4a-4jb69"]
Sep 14 08:07:17.381747 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:17.381660    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:17.575154 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:17.575117    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:17.793640 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:17.793605    2562 generic.go:356] "Generic (PLEG): container finished" podID="896de534-987b-452c-a3a3-960f67ed3cc8" containerID="331fea5c716ada12f03eaee4b5f0fb7a548b2ad03659ae20b54f2670df9e70db" exitCode=0
Sep 14 08:07:17.793816 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:17.793682    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878" event={"ID":"896de534-987b-452c-a3a3-960f67ed3cc8","Type":"ContainerDied","Data":"331fea5c716ada12f03eaee4b5f0fb7a548b2ad03659ae20b54f2670df9e70db"}
Sep 14 08:07:17.794147 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:17.794130    2562 scope.go:117] "RemoveContainer" containerID="331fea5c716ada12f03eaee4b5f0fb7a548b2ad03659ae20b54f2670df9e70db"
Sep 14 08:07:17.795281 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:17.795254    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:07:17.795379 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:17.795354    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:07:17.795445 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:17.795406    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:07:18.053887 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:18.053778    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:18.117827 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:18.117564    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72df8032-0167-4db0-b21a-7aebdba43d98" path="/var/lib/kubelet/pods/72df8032-0167-4db0-b21a-7aebdba43d98/volumes"
Sep 14 08:07:18.136827 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:18.136793    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:18.195780 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:18.195746    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:18.296399 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:18.296357    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:18.356729 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:18.356620    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:18.593518 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:18.593479    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:18.738264 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:18.738220    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:18.801022 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:18.800988    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878" event={"ID":"896de534-987b-452c-a3a3-960f67ed3cc8","Type":"ContainerStarted","Data":"21f9fec4953c939dc7f93d1a482ba06599a1bbbf630d6f9b41832c69bd4a973d"}
Sep 14 08:07:19.390458 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:19.390417    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:07:19.391512 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:19.391478    2562 patch_prober.go:28] interesting pod/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 14 08:07:19.391640 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:19.391547    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878" podUID="896de534-987b-452c-a3a3-960f67ed3cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 14 08:07:19.426405 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:19.426373    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:19.621686 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:19.621644    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:07:19.643223 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:19.643128    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:19.930630 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:19.930539    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:20.030483 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:20.030419    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:20.221703 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:20.221672    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:20.390598 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:20.390564    2562 patch_prober.go:28] interesting pod/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 14 08:07:20.390993 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:20.390629    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878" podUID="896de534-987b-452c-a3a3-960f67ed3cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 14 08:07:20.712673 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:20.712632    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv" podUID="aba992c0-2f6e-4b97-b69e-7194f3e3be7c" containerName="main" containerID="cri-o://768f17a406300abad23995365588e1f8bf206d869ea3792b22ee378d07c2f48d" gracePeriod=45
Sep 14 08:07:20.931209 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:20.931171    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:21.017196 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.017164    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:07:21.099193 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.099157    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-tls-certs\") pod \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") "
Sep 14 08:07:21.099193 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.099199    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-model-cache\") pod \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") "
Sep 14 08:07:21.099441 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.099263    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-dshm\") pod \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") "
Sep 14 08:07:21.099441 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.099293    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lq48j\" (UniqueName: \"kubernetes.io/projected/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-kube-api-access-lq48j\") pod \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") "
Sep 14 08:07:21.099441 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.099315    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-home\") pod \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") "
Sep 14 08:07:21.099441 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.099332    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-tmp-dir\") pod \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\" (UID: \"aba992c0-2f6e-4b97-b69e-7194f3e3be7c\") "
Sep 14 08:07:21.099708 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.099536    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-model-cache" (OuterVolumeSpecName: "model-cache") pod "aba992c0-2f6e-4b97-b69e-7194f3e3be7c" (UID: "aba992c0-2f6e-4b97-b69e-7194f3e3be7c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:21.099767 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.099703    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-home" (OuterVolumeSpecName: "home") pod "aba992c0-2f6e-4b97-b69e-7194f3e3be7c" (UID: "aba992c0-2f6e-4b97-b69e-7194f3e3be7c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:21.099767 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.099719    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aba992c0-2f6e-4b97-b69e-7194f3e3be7c" (UID: "aba992c0-2f6e-4b97-b69e-7194f3e3be7c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:21.099767 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.099739    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-model-cache\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:21.101560 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.101533    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-kube-api-access-lq48j" (OuterVolumeSpecName: "kube-api-access-lq48j") pod "aba992c0-2f6e-4b97-b69e-7194f3e3be7c" (UID: "aba992c0-2f6e-4b97-b69e-7194f3e3be7c"). InnerVolumeSpecName "kube-api-access-lq48j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:21.101560 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.101555    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aba992c0-2f6e-4b97-b69e-7194f3e3be7c" (UID: "aba992c0-2f6e-4b97-b69e-7194f3e3be7c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:07:21.101742 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.101606    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-dshm" (OuterVolumeSpecName: "dshm") pod "aba992c0-2f6e-4b97-b69e-7194f3e3be7c" (UID: "aba992c0-2f6e-4b97-b69e-7194f3e3be7c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:21.200801 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.200771    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-tls-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:21.200801 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.200796    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-dshm\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:21.200801 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.200805    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lq48j\" (UniqueName: \"kubernetes.io/projected/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-kube-api-access-lq48j\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:21.201094 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.200815    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-home\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:21.201094 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.200830    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aba992c0-2f6e-4b97-b69e-7194f3e3be7c-tmp-dir\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:21.390730 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.390641    2562 patch_prober.go:28] interesting pod/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 14 08:07:21.390730 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.390705    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878" podUID="896de534-987b-452c-a3a3-960f67ed3cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 14 08:07:21.753343 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.753269    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:21.818899 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.818861    2562 generic.go:356] "Generic (PLEG): container finished" podID="aba992c0-2f6e-4b97-b69e-7194f3e3be7c" containerID="768f17a406300abad23995365588e1f8bf206d869ea3792b22ee378d07c2f48d" exitCode=0
Sep 14 08:07:21.819103 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.818913    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv" event={"ID":"aba992c0-2f6e-4b97-b69e-7194f3e3be7c","Type":"ContainerDied","Data":"768f17a406300abad23995365588e1f8bf206d869ea3792b22ee378d07c2f48d"}
Sep 14 08:07:21.819103 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.818921    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"
Sep 14 08:07:21.819103 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.818953    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv" event={"ID":"aba992c0-2f6e-4b97-b69e-7194f3e3be7c","Type":"ContainerDied","Data":"a4a6cc30c396069ced37534ce9ee170e72b796a03ae65041e19065f060e91f2d"}
Sep 14 08:07:21.819103 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.818978    2562 scope.go:117] "RemoveContainer" containerID="768f17a406300abad23995365588e1f8bf206d869ea3792b22ee378d07c2f48d"
Sep 14 08:07:21.828484 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.828461    2562 scope.go:117] "RemoveContainer" containerID="768f17a406300abad23995365588e1f8bf206d869ea3792b22ee378d07c2f48d"
Sep 14 08:07:21.828819 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:07:21.828798    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"768f17a406300abad23995365588e1f8bf206d869ea3792b22ee378d07c2f48d\": container with ID starting with 768f17a406300abad23995365588e1f8bf206d869ea3792b22ee378d07c2f48d not found: ID does not exist" containerID="768f17a406300abad23995365588e1f8bf206d869ea3792b22ee378d07c2f48d"
Sep 14 08:07:21.828890 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.828831    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"768f17a406300abad23995365588e1f8bf206d869ea3792b22ee378d07c2f48d"} err="failed to get container status \"768f17a406300abad23995365588e1f8bf206d869ea3792b22ee378d07c2f48d\": rpc error: code = NotFound desc = could not find container \"768f17a406300abad23995365588e1f8bf206d869ea3792b22ee378d07c2f48d\": container with ID starting with 768f17a406300abad23995365588e1f8bf206d869ea3792b22ee378d07c2f48d not found: ID does not exist"
Sep 14 08:07:21.845161 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.845124    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"]
Sep 14 08:07:21.850006 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.849966    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-73bd4a/explicit-model-73bd4a-kserve-596f74bbf9-xxhnv"]
Sep 14 08:07:21.907958 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:21.907919    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr" podUID="6d91878b-c89b-4d05-8efa-6836e3ab8872" containerName="main" containerID="cri-o://0599e4a7074d2cad3cc7fb771cd649c084b3b664c0f88704e0fb8fb4b8e8919a" gracePeriod=45
Sep 14 08:07:22.116764 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.116730    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aba992c0-2f6e-4b97-b69e-7194f3e3be7c" path="/var/lib/kubelet/pods/aba992c0-2f6e-4b97-b69e-7194f3e3be7c/volumes"
Sep 14 08:07:22.192260 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.192236    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:07:22.312022 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.311931    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d91878b-c89b-4d05-8efa-6836e3ab8872-tls-certs\") pod \"6d91878b-c89b-4d05-8efa-6836e3ab8872\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") "
Sep 14 08:07:22.312022 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.311989    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-tmp-dir\") pod \"6d91878b-c89b-4d05-8efa-6836e3ab8872\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") "
Sep 14 08:07:22.312222 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.312030    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4tkc\" (UniqueName: \"kubernetes.io/projected/6d91878b-c89b-4d05-8efa-6836e3ab8872-kube-api-access-j4tkc\") pod \"6d91878b-c89b-4d05-8efa-6836e3ab8872\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") "
Sep 14 08:07:22.312222 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.312060    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-dshm\") pod \"6d91878b-c89b-4d05-8efa-6836e3ab8872\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") "
Sep 14 08:07:22.312222 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.312117    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-model-cache\") pod \"6d91878b-c89b-4d05-8efa-6836e3ab8872\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") "
Sep 14 08:07:22.312222 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.312157    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-home\") pod \"6d91878b-c89b-4d05-8efa-6836e3ab8872\" (UID: \"6d91878b-c89b-4d05-8efa-6836e3ab8872\") "
Sep 14 08:07:22.312456 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.312270    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6d91878b-c89b-4d05-8efa-6836e3ab8872" (UID: "6d91878b-c89b-4d05-8efa-6836e3ab8872"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:22.312456 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.312342    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-model-cache" (OuterVolumeSpecName: "model-cache") pod "6d91878b-c89b-4d05-8efa-6836e3ab8872" (UID: "6d91878b-c89b-4d05-8efa-6836e3ab8872"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:22.312456 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.312433    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-home" (OuterVolumeSpecName: "home") pod "6d91878b-c89b-4d05-8efa-6836e3ab8872" (UID: "6d91878b-c89b-4d05-8efa-6836e3ab8872"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:22.312645 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.312544    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-tmp-dir\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:22.312645 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.312561    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-model-cache\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:22.314271 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.314246    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d91878b-c89b-4d05-8efa-6836e3ab8872-kube-api-access-j4tkc" (OuterVolumeSpecName: "kube-api-access-j4tkc") pod "6d91878b-c89b-4d05-8efa-6836e3ab8872" (UID: "6d91878b-c89b-4d05-8efa-6836e3ab8872"). InnerVolumeSpecName "kube-api-access-j4tkc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:22.314271 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.314264    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-dshm" (OuterVolumeSpecName: "dshm") pod "6d91878b-c89b-4d05-8efa-6836e3ab8872" (UID: "6d91878b-c89b-4d05-8efa-6836e3ab8872"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:22.314415 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.314270    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d91878b-c89b-4d05-8efa-6836e3ab8872-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6d91878b-c89b-4d05-8efa-6836e3ab8872" (UID: "6d91878b-c89b-4d05-8efa-6836e3ab8872"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:07:22.390440 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.390401    2562 patch_prober.go:28] interesting pod/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 14 08:07:22.390645 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.390461    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878" podUID="896de534-987b-452c-a3a3-960f67ed3cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 14 08:07:22.413620 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.413577    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-home\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:22.413620 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.413611    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d91878b-c89b-4d05-8efa-6836e3ab8872-tls-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:22.413620 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.413627    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4tkc\" (UniqueName: \"kubernetes.io/projected/6d91878b-c89b-4d05-8efa-6836e3ab8872-kube-api-access-j4tkc\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:22.414059 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.413638    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d91878b-c89b-4d05-8efa-6836e3ab8872-dshm\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:22.619457 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.619354    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:22.826441 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.826404    2562 generic.go:356] "Generic (PLEG): container finished" podID="6d91878b-c89b-4d05-8efa-6836e3ab8872" containerID="0599e4a7074d2cad3cc7fb771cd649c084b3b664c0f88704e0fb8fb4b8e8919a" exitCode=0
Sep 14 08:07:22.826651 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.826447    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"
Sep 14 08:07:22.826651 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.826494    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr" event={"ID":"6d91878b-c89b-4d05-8efa-6836e3ab8872","Type":"ContainerDied","Data":"0599e4a7074d2cad3cc7fb771cd649c084b3b664c0f88704e0fb8fb4b8e8919a"}
Sep 14 08:07:22.826651 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.826567    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr" event={"ID":"6d91878b-c89b-4d05-8efa-6836e3ab8872","Type":"ContainerDied","Data":"c87e92dee75895b8a24a59390d8b563e0ea17b16ec85940f781bc17c87f61c39"}
Sep 14 08:07:22.826651 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.826590    2562 scope.go:117] "RemoveContainer" containerID="0599e4a7074d2cad3cc7fb771cd649c084b3b664c0f88704e0fb8fb4b8e8919a"
Sep 14 08:07:22.835795 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.835775    2562 scope.go:117] "RemoveContainer" containerID="0599e4a7074d2cad3cc7fb771cd649c084b3b664c0f88704e0fb8fb4b8e8919a"
Sep 14 08:07:22.836078 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:07:22.836061    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0599e4a7074d2cad3cc7fb771cd649c084b3b664c0f88704e0fb8fb4b8e8919a\": container with ID starting with 0599e4a7074d2cad3cc7fb771cd649c084b3b664c0f88704e0fb8fb4b8e8919a not found: ID does not exist" containerID="0599e4a7074d2cad3cc7fb771cd649c084b3b664c0f88704e0fb8fb4b8e8919a"
Sep 14 08:07:22.836121 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.836086    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0599e4a7074d2cad3cc7fb771cd649c084b3b664c0f88704e0fb8fb4b8e8919a"} err="failed to get container status \"0599e4a7074d2cad3cc7fb771cd649c084b3b664c0f88704e0fb8fb4b8e8919a\": rpc error: code = NotFound desc = could not find container \"0599e4a7074d2cad3cc7fb771cd649c084b3b664c0f88704e0fb8fb4b8e8919a\": container with ID starting with 0599e4a7074d2cad3cc7fb771cd649c084b3b664c0f88704e0fb8fb4b8e8919a not found: ID does not exist"
Sep 14 08:07:22.847625 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.847590    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"]
Sep 14 08:07:22.851569 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:22.851535    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-5ae4e6/test-model-5ae4e6-kserve-85c844bb57-mb8qr"]
Sep 14 08:07:23.353344 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:23.353301    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:23.390577 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:23.390541    2562 patch_prober.go:28] interesting pod/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 14 08:07:23.390759 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:23.390603    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878" podUID="896de534-987b-452c-a3a3-960f67ed3cc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 14 08:07:23.448239 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:23.448197    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:24.113325 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:24.113281    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d91878b-c89b-4d05-8efa-6836e3ab8872" path="/var/lib/kubelet/pods/6d91878b-c89b-4d05-8efa-6836e3ab8872/volumes"
Sep 14 08:07:24.205650 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:24.205618    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:24.310323 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:24.310290    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:24.395858 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:24.395777    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:07:24.397097 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:24.397068    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:07:24.406205 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:24.406175    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:24.412025 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:24.411994    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"]
Sep 14 08:07:24.412255 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:24.412232    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s" podUID="fef6e286-c65d-46bd-a85f-aa6eca82dbaa" containerName="istio-proxy" containerID="cri-o://42d0ebf246b5388c59de929e77c942f35dd572bc4117d90a8e5741f2d8613b79" gracePeriod=30
Sep 14 08:07:24.505630 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:24.505598    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:24.856415 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:24.856378    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:25.157615 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.157540    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"]
Sep 14 08:07:25.157960 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.157947    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72df8032-0167-4db0-b21a-7aebdba43d98" containerName="revoke-keys"
Sep 14 08:07:25.158023 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.157963    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="72df8032-0167-4db0-b21a-7aebdba43d98" containerName="revoke-keys"
Sep 14 08:07:25.158023 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.157972    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d91878b-c89b-4d05-8efa-6836e3ab8872" containerName="main"
Sep 14 08:07:25.158023 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.157978    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d91878b-c89b-4d05-8efa-6836e3ab8872" containerName="main"
Sep 14 08:07:25.158023 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.157993    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aba992c0-2f6e-4b97-b69e-7194f3e3be7c" containerName="main"
Sep 14 08:07:25.158023 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.157999    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="aba992c0-2f6e-4b97-b69e-7194f3e3be7c" containerName="main"
Sep 14 08:07:25.158197 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.158071    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d91878b-c89b-4d05-8efa-6836e3ab8872" containerName="main"
Sep 14 08:07:25.158197 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.158079    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="72df8032-0167-4db0-b21a-7aebdba43d98" containerName="revoke-keys"
Sep 14 08:07:25.158197 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.158088    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="aba992c0-2f6e-4b97-b69e-7194f3e3be7c" containerName="main"
Sep 14 08:07:25.162437 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.162407    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.165273 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.165250    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-48e079-openshift-default-dockercfg-t5kdm\""
Sep 14 08:07:25.178762 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.178739    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"]
Sep 14 08:07:25.182873 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.182837    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:25.241793 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.241760    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-workload-socket\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.241793 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.241804    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-credential-socket\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.242067 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.241870    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-istio-envoy\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.242067 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.241916    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/056170e3-8e2a-450f-936d-29fd341e1d79-istio-podinfo\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.242067 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.241972    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldkbm\" (UniqueName: \"kubernetes.io/projected/056170e3-8e2a-450f-936d-29fd341e1d79-kube-api-access-ldkbm\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.242067 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.241996    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-workload-certs\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.242067 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.242016    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/056170e3-8e2a-450f-936d-29fd341e1d79-istio-token\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.242324 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.242086    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-istio-data\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.242324 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.242156    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/056170e3-8e2a-450f-936d-29fd341e1d79-istiod-ca-cert\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.254914 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.254879    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:25.343523 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.343357    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldkbm\" (UniqueName: \"kubernetes.io/projected/056170e3-8e2a-450f-936d-29fd341e1d79-kube-api-access-ldkbm\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.343523 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.343417    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-workload-certs\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.343523 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.343456    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/056170e3-8e2a-450f-936d-29fd341e1d79-istio-token\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.343818 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.343552    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-istio-data\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.343818 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.343634    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/056170e3-8e2a-450f-936d-29fd341e1d79-istiod-ca-cert\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.343818 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.343687    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-workload-socket\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.343818 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.343725    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-credential-socket\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.343818 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.343771    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-istio-envoy\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.343818 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.343798    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/056170e3-8e2a-450f-936d-29fd341e1d79-istio-podinfo\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.347517 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.344844    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/056170e3-8e2a-450f-936d-29fd341e1d79-istiod-ca-cert\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.347517 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.344971    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-workload-certs\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.347517 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.345164    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-istio-data\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.347517 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.345232    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-credential-socket\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.347517 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.345476    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-workload-socket\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.349524 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.348638    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/056170e3-8e2a-450f-936d-29fd341e1d79-istio-podinfo\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.355287 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.354233    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-istio-envoy\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.359011 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.358972    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/056170e3-8e2a-450f-936d-29fd341e1d79-istio-token\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.360657 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.360609    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldkbm\" (UniqueName: \"kubernetes.io/projected/056170e3-8e2a-450f-936d-29fd341e1d79-kube-api-access-ldkbm\") pod \"e2e-noten-48e079-openshift-default-5989d455cd-gzljd\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") " pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.473324 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.473267    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:25.489613 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.489570    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:25.750653 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.750625    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"]
Sep 14 08:07:25.753347 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:07:25.753321    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod056170e3_8e2a_450f_936d_29fd341e1d79.slice/crio-6730d061620717971f67520968cf33acb8b456dc958d473c3cca7024ca0562d4 WatchSource:0}: Error finding container 6730d061620717971f67520968cf33acb8b456dc958d473c3cca7024ca0562d4: Status 404 returned error can't find the container with id 6730d061620717971f67520968cf33acb8b456dc958d473c3cca7024ca0562d4
Sep 14 08:07:25.755907 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.755879    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:07:25.755978 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.755941    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:07:25.755978 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.755970    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:07:25.847137 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.847101    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd" event={"ID":"056170e3-8e2a-450f-936d-29fd341e1d79","Type":"ContainerStarted","Data":"04573897bfda9ddc72d0454d1163c66da9453254a2f4a8efaba1685d006998e2"}
Sep 14 08:07:25.847322 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.847164    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd" event={"ID":"056170e3-8e2a-450f-936d-29fd341e1d79","Type":"ContainerStarted","Data":"6730d061620717971f67520968cf33acb8b456dc958d473c3cca7024ca0562d4"}
Sep 14 08:07:25.868935 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.868865    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd" podStartSLOduration=0.868847973 podStartE2EDuration="868.847973ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:25.866640111 +0000 UTC m=+1148.323090107" watchObservedRunningTime="2026-09-14 08:07:25.868847973 +0000 UTC m=+1148.325297960"
Sep 14 08:07:25.955107 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:25.955069    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:26.055812 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:26.055715    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:26.155952 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:26.155912    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:26.176993 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:26.176960    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"]
Sep 14 08:07:26.177238 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:26.177213    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq" podUID="f0d3cbc9-be52-4983-901e-76e256c6abae" containerName="istio-proxy" containerID="cri-o://987b61705e37f5fdff6f2cbd5d02cf18541ed5145501dcd6e1f33b6d20c4f7e4" gracePeriod=30
Sep 14 08:07:26.255438 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:26.255403    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:26.360276 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:26.360151    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:26.474397 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:26.474357    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:26.479457 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:26.479421    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:26.485068 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:26.485028    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:26.649372 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:26.649290    2562 patch_prober.go:28] interesting pod/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 08:07:26.649372 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:26.649351    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s" podUID="fef6e286-c65d-46bd-a85f-aa6eca82dbaa" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 08:07:26.852217 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:26.852184    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:26.853043 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:26.853020    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:07:27.132228 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:27.132193    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:27.913707 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:27.913672    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:28.017397 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:28.017364    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:28.114387 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:28.114345    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:28.413927 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:28.413893    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:28.472754 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:28.472720    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:28.827423 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:28.827390    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:29.621048 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.621007    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:29.670741 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.670715    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:07:29.795635 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.795598    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pw86f\" (UniqueName: \"kubernetes.io/projected/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-kube-api-access-pw86f\") pod \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") "
Sep 14 08:07:29.795839 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.795669    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-credential-socket\") pod \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") "
Sep 14 08:07:29.795839 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.795694    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-data\") pod \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") "
Sep 14 08:07:29.795839 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.795716    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-podinfo\") pod \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") "
Sep 14 08:07:29.795839 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.795732    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-token\") pod \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") "
Sep 14 08:07:29.795839 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.795759    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-workload-socket\") pod \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") "
Sep 14 08:07:29.795839 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.795780    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istiod-ca-cert\") pod \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") "
Sep 14 08:07:29.796145 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.795914    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-envoy\") pod \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") "
Sep 14 08:07:29.796145 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.795938    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-workload-certs\") pod \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\" (UID: \"fef6e286-c65d-46bd-a85f-aa6eca82dbaa\") "
Sep 14 08:07:29.796145 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.795941    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fef6e286-c65d-46bd-a85f-aa6eca82dbaa" (UID: "fef6e286-c65d-46bd-a85f-aa6eca82dbaa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:29.796145 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.796057    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-data" (OuterVolumeSpecName: "istio-data") pod "fef6e286-c65d-46bd-a85f-aa6eca82dbaa" (UID: "fef6e286-c65d-46bd-a85f-aa6eca82dbaa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:29.796145 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.796102    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fef6e286-c65d-46bd-a85f-aa6eca82dbaa" (UID: "fef6e286-c65d-46bd-a85f-aa6eca82dbaa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:29.796397 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.796287    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fef6e286-c65d-46bd-a85f-aa6eca82dbaa" (UID: "fef6e286-c65d-46bd-a85f-aa6eca82dbaa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:29.796397 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.796378    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-credential-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:29.796492 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.796399    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-data\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:29.796492 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.796415    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-workload-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:29.796492 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.796428    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istiod-ca-cert\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:29.796492 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.796376    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fef6e286-c65d-46bd-a85f-aa6eca82dbaa" (UID: "fef6e286-c65d-46bd-a85f-aa6eca82dbaa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:29.798565 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.798541    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fef6e286-c65d-46bd-a85f-aa6eca82dbaa" (UID: "fef6e286-c65d-46bd-a85f-aa6eca82dbaa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:29.798565 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.798565    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fef6e286-c65d-46bd-a85f-aa6eca82dbaa" (UID: "fef6e286-c65d-46bd-a85f-aa6eca82dbaa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:07:29.798691 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.798586    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-kube-api-access-pw86f" (OuterVolumeSpecName: "kube-api-access-pw86f") pod "fef6e286-c65d-46bd-a85f-aa6eca82dbaa" (UID: "fef6e286-c65d-46bd-a85f-aa6eca82dbaa"). InnerVolumeSpecName "kube-api-access-pw86f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:29.798691 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.798601    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-token" (OuterVolumeSpecName: "istio-token") pod "fef6e286-c65d-46bd-a85f-aa6eca82dbaa" (UID: "fef6e286-c65d-46bd-a85f-aa6eca82dbaa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:29.869205 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.869169    2562 generic.go:356] "Generic (PLEG): container finished" podID="fef6e286-c65d-46bd-a85f-aa6eca82dbaa" containerID="42d0ebf246b5388c59de929e77c942f35dd572bc4117d90a8e5741f2d8613b79" exitCode=0
Sep 14 08:07:29.869381 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.869213    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"
Sep 14 08:07:29.869381 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.869246    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s" event={"ID":"fef6e286-c65d-46bd-a85f-aa6eca82dbaa","Type":"ContainerDied","Data":"42d0ebf246b5388c59de929e77c942f35dd572bc4117d90a8e5741f2d8613b79"}
Sep 14 08:07:29.869381 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.869298    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s" event={"ID":"fef6e286-c65d-46bd-a85f-aa6eca82dbaa","Type":"ContainerDied","Data":"749fbd4fc7c87137953a6345aece5cf8e542a0107c8fe5cd0ceda7fdadd2c080"}
Sep 14 08:07:29.869381 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.869320    2562 scope.go:117] "RemoveContainer" containerID="42d0ebf246b5388c59de929e77c942f35dd572bc4117d90a8e5741f2d8613b79"
Sep 14 08:07:29.878960 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.878942    2562 scope.go:117] "RemoveContainer" containerID="91bbf91cd2588c4d50f1ec86683f95b842e0a374c5ce2f7f75e045e890dcace4"
Sep 14 08:07:29.887352 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.887324    2562 scope.go:117] "RemoveContainer" containerID="42d0ebf246b5388c59de929e77c942f35dd572bc4117d90a8e5741f2d8613b79"
Sep 14 08:07:29.887652 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:07:29.887632    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42d0ebf246b5388c59de929e77c942f35dd572bc4117d90a8e5741f2d8613b79\": container with ID starting with 42d0ebf246b5388c59de929e77c942f35dd572bc4117d90a8e5741f2d8613b79 not found: ID does not exist" containerID="42d0ebf246b5388c59de929e77c942f35dd572bc4117d90a8e5741f2d8613b79"
Sep 14 08:07:29.887706 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.887661    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42d0ebf246b5388c59de929e77c942f35dd572bc4117d90a8e5741f2d8613b79"} err="failed to get container status \"42d0ebf246b5388c59de929e77c942f35dd572bc4117d90a8e5741f2d8613b79\": rpc error: code = NotFound desc = could not find container \"42d0ebf246b5388c59de929e77c942f35dd572bc4117d90a8e5741f2d8613b79\": container with ID starting with 42d0ebf246b5388c59de929e77c942f35dd572bc4117d90a8e5741f2d8613b79 not found: ID does not exist"
Sep 14 08:07:29.887706 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.887678    2562 scope.go:117] "RemoveContainer" containerID="91bbf91cd2588c4d50f1ec86683f95b842e0a374c5ce2f7f75e045e890dcace4"
Sep 14 08:07:29.887938 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:07:29.887921    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91bbf91cd2588c4d50f1ec86683f95b842e0a374c5ce2f7f75e045e890dcace4\": container with ID starting with 91bbf91cd2588c4d50f1ec86683f95b842e0a374c5ce2f7f75e045e890dcace4 not found: ID does not exist" containerID="91bbf91cd2588c4d50f1ec86683f95b842e0a374c5ce2f7f75e045e890dcace4"
Sep 14 08:07:29.887981 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.887944    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91bbf91cd2588c4d50f1ec86683f95b842e0a374c5ce2f7f75e045e890dcace4"} err="failed to get container status \"91bbf91cd2588c4d50f1ec86683f95b842e0a374c5ce2f7f75e045e890dcace4\": rpc error: code = NotFound desc = could not find container \"91bbf91cd2588c4d50f1ec86683f95b842e0a374c5ce2f7f75e045e890dcace4\": container with ID starting with 91bbf91cd2588c4d50f1ec86683f95b842e0a374c5ce2f7f75e045e890dcace4 not found: ID does not exist"
Sep 14 08:07:29.891348 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.891327    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"]
Sep 14 08:07:29.895340 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.895319    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-73bd4a-openshift-default-65b88c8b4-jdj7s"]
Sep 14 08:07:29.897373 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.897356    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-podinfo\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:29.897431 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.897378    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-token\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:29.897431 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.897386    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-istio-envoy\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:29.897431 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.897394    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-workload-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:29.897431 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:29.897402    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pw86f\" (UniqueName: \"kubernetes.io/projected/fef6e286-c65d-46bd-a85f-aa6eca82dbaa-kube-api-access-pw86f\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:30.112766 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:30.112686    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fef6e286-c65d-46bd-a85f-aa6eca82dbaa" path="/var/lib/kubelet/pods/fef6e286-c65d-46bd-a85f-aa6eca82dbaa/volumes"
Sep 14 08:07:30.540482 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:30.540438    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:31.235311 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.235274    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:31.443031 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.442963    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:07:31.517659 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.517619    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rbws\" (UniqueName: \"kubernetes.io/projected/f0d3cbc9-be52-4983-901e-76e256c6abae-kube-api-access-4rbws\") pod \"f0d3cbc9-be52-4983-901e-76e256c6abae\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") "
Sep 14 08:07:31.517857 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.517667    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-workload-certs\") pod \"f0d3cbc9-be52-4983-901e-76e256c6abae\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") "
Sep 14 08:07:31.517857 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.517711    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-credential-socket\") pod \"f0d3cbc9-be52-4983-901e-76e256c6abae\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") "
Sep 14 08:07:31.517857 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.517787    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-podinfo\") pod \"f0d3cbc9-be52-4983-901e-76e256c6abae\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") "
Sep 14 08:07:31.517857 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.517810    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-data\") pod \"f0d3cbc9-be52-4983-901e-76e256c6abae\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") "
Sep 14 08:07:31.518086 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.517865    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-workload-socket\") pod \"f0d3cbc9-be52-4983-901e-76e256c6abae\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") "
Sep 14 08:07:31.518086 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.517889    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0d3cbc9-be52-4983-901e-76e256c6abae-istiod-ca-cert\") pod \"f0d3cbc9-be52-4983-901e-76e256c6abae\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") "
Sep 14 08:07:31.518086 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.517950    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-envoy\") pod \"f0d3cbc9-be52-4983-901e-76e256c6abae\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") "
Sep 14 08:07:31.518086 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.517982    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-token\") pod \"f0d3cbc9-be52-4983-901e-76e256c6abae\" (UID: \"f0d3cbc9-be52-4983-901e-76e256c6abae\") "
Sep 14 08:07:31.518086 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.518038    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f0d3cbc9-be52-4983-901e-76e256c6abae" (UID: "f0d3cbc9-be52-4983-901e-76e256c6abae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:31.518350 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.518084    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f0d3cbc9-be52-4983-901e-76e256c6abae" (UID: "f0d3cbc9-be52-4983-901e-76e256c6abae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:31.518350 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.518096    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f0d3cbc9-be52-4983-901e-76e256c6abae" (UID: "f0d3cbc9-be52-4983-901e-76e256c6abae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:31.518350 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.518107    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-data" (OuterVolumeSpecName: "istio-data") pod "f0d3cbc9-be52-4983-901e-76e256c6abae" (UID: "f0d3cbc9-be52-4983-901e-76e256c6abae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:31.518350 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.518299    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0d3cbc9-be52-4983-901e-76e256c6abae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f0d3cbc9-be52-4983-901e-76e256c6abae" (UID: "f0d3cbc9-be52-4983-901e-76e256c6abae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:31.518581 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.518378    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-data\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:31.518581 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.518397    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-workload-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:31.518581 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.518411    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0d3cbc9-be52-4983-901e-76e256c6abae-istiod-ca-cert\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:31.518581 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.518425    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-workload-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:31.518581 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.518438    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-credential-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:31.520028 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.519993    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f0d3cbc9-be52-4983-901e-76e256c6abae" (UID: "f0d3cbc9-be52-4983-901e-76e256c6abae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:07:31.520160 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.520141    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0d3cbc9-be52-4983-901e-76e256c6abae-kube-api-access-4rbws" (OuterVolumeSpecName: "kube-api-access-4rbws") pod "f0d3cbc9-be52-4983-901e-76e256c6abae" (UID: "f0d3cbc9-be52-4983-901e-76e256c6abae"). InnerVolumeSpecName "kube-api-access-4rbws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:31.520449 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.520435    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f0d3cbc9-be52-4983-901e-76e256c6abae" (UID: "f0d3cbc9-be52-4983-901e-76e256c6abae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:31.520543 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.520470    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-token" (OuterVolumeSpecName: "istio-token") pod "f0d3cbc9-be52-4983-901e-76e256c6abae" (UID: "f0d3cbc9-be52-4983-901e-76e256c6abae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:31.619422 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.619377    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-podinfo\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:31.619422 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.619414    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-envoy\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:31.619422 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.619426    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0d3cbc9-be52-4983-901e-76e256c6abae-istio-token\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:31.619698 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.619439    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rbws\" (UniqueName: \"kubernetes.io/projected/f0d3cbc9-be52-4983-901e-76e256c6abae-kube-api-access-4rbws\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:31.883844 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.883750    2562 generic.go:356] "Generic (PLEG): container finished" podID="f0d3cbc9-be52-4983-901e-76e256c6abae" containerID="987b61705e37f5fdff6f2cbd5d02cf18541ed5145501dcd6e1f33b6d20c4f7e4" exitCode=0
Sep 14 08:07:31.883844 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.883797    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"
Sep 14 08:07:31.883844 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.883826    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq" event={"ID":"f0d3cbc9-be52-4983-901e-76e256c6abae","Type":"ContainerDied","Data":"987b61705e37f5fdff6f2cbd5d02cf18541ed5145501dcd6e1f33b6d20c4f7e4"}
Sep 14 08:07:31.884068 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.883901    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq" event={"ID":"f0d3cbc9-be52-4983-901e-76e256c6abae","Type":"ContainerDied","Data":"4ee791db6c86228c23f6dae7943e92d0c6aa199503379da59139e297134898c5"}
Sep 14 08:07:31.884068 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.883942    2562 scope.go:117] "RemoveContainer" containerID="987b61705e37f5fdff6f2cbd5d02cf18541ed5145501dcd6e1f33b6d20c4f7e4"
Sep 14 08:07:31.893614 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.893594    2562 scope.go:117] "RemoveContainer" containerID="d2ca80d73a73a476743524dc1cc539ab59c29bae6d726251b646214bd6ff245b"
Sep 14 08:07:31.901977 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.901958    2562 scope.go:117] "RemoveContainer" containerID="987b61705e37f5fdff6f2cbd5d02cf18541ed5145501dcd6e1f33b6d20c4f7e4"
Sep 14 08:07:31.902273 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:07:31.902250    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"987b61705e37f5fdff6f2cbd5d02cf18541ed5145501dcd6e1f33b6d20c4f7e4\": container with ID starting with 987b61705e37f5fdff6f2cbd5d02cf18541ed5145501dcd6e1f33b6d20c4f7e4 not found: ID does not exist" containerID="987b61705e37f5fdff6f2cbd5d02cf18541ed5145501dcd6e1f33b6d20c4f7e4"
Sep 14 08:07:31.902315 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.902283    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"987b61705e37f5fdff6f2cbd5d02cf18541ed5145501dcd6e1f33b6d20c4f7e4"} err="failed to get container status \"987b61705e37f5fdff6f2cbd5d02cf18541ed5145501dcd6e1f33b6d20c4f7e4\": rpc error: code = NotFound desc = could not find container \"987b61705e37f5fdff6f2cbd5d02cf18541ed5145501dcd6e1f33b6d20c4f7e4\": container with ID starting with 987b61705e37f5fdff6f2cbd5d02cf18541ed5145501dcd6e1f33b6d20c4f7e4 not found: ID does not exist"
Sep 14 08:07:31.902315 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.902299    2562 scope.go:117] "RemoveContainer" containerID="d2ca80d73a73a476743524dc1cc539ab59c29bae6d726251b646214bd6ff245b"
Sep 14 08:07:31.902578 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:07:31.902560    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2ca80d73a73a476743524dc1cc539ab59c29bae6d726251b646214bd6ff245b\": container with ID starting with d2ca80d73a73a476743524dc1cc539ab59c29bae6d726251b646214bd6ff245b not found: ID does not exist" containerID="d2ca80d73a73a476743524dc1cc539ab59c29bae6d726251b646214bd6ff245b"
Sep 14 08:07:31.902629 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.902585    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ca80d73a73a476743524dc1cc539ab59c29bae6d726251b646214bd6ff245b"} err="failed to get container status \"d2ca80d73a73a476743524dc1cc539ab59c29bae6d726251b646214bd6ff245b\": rpc error: code = NotFound desc = could not find container \"d2ca80d73a73a476743524dc1cc539ab59c29bae6d726251b646214bd6ff245b\": container with ID starting with d2ca80d73a73a476743524dc1cc539ab59c29bae6d726251b646214bd6ff245b not found: ID does not exist"
Sep 14 08:07:31.909779 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.909754    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"]
Sep 14 08:07:31.913459 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.913437    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-3a842e-openshift-default-9c65f876-cp9zq"]
Sep 14 08:07:31.934092 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:31.934062    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:32.112956 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:32.112924    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0d3cbc9-be52-4983-901e-76e256c6abae" path="/var/lib/kubelet/pods/f0d3cbc9-be52-4983-901e-76e256c6abae/volumes"
Sep 14 08:07:33.288366 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:33.288320    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:33.345051 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:33.345015    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:36.844425 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:36.844371    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:37.450465 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:37.450409    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:37.518390 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:37.518353    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:37.577007 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:37.576964    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:37.659586 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:37.659553    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:37.718375 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:37.718262    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:38.015422 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:38.015327    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:38.016737 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:38.016710    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:07:38.118032 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:38.117980    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:38.939339 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:38.939288    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:39.663111 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:39.663076    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:40.536557 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:40.536472    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:41.400904 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:41.400865    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:42.249021 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:42.248986    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:42.956042 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:42.956002    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:43.185487 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:43.185454    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:44.263389 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:44.263348    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:44.363911 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:44.363871    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:44.464649 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:44.464606    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:44.577134 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:44.577045    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"]
Sep 14 08:07:44.577287 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:44.577265    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt" podUID="8512cd08-a696-4695-9bbf-c6c3d68d7329" containerName="istio-proxy" containerID="cri-o://d17232d3dd3ad19f8744ad693a56d330898dabec149cd26fd250c1621a12f15b" gracePeriod=30
Sep 14 08:07:44.611178 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:44.611140    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:44.712176 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:44.712134    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:45.154326 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.154294    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:45.427568 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.427476    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"]
Sep 14 08:07:45.428070 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.428048    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fef6e286-c65d-46bd-a85f-aa6eca82dbaa" containerName="istio-proxy"
Sep 14 08:07:45.428148 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.428073    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fef6e286-c65d-46bd-a85f-aa6eca82dbaa" containerName="istio-proxy"
Sep 14 08:07:45.428148 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.428090    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fef6e286-c65d-46bd-a85f-aa6eca82dbaa" containerName="istio-proxy"
Sep 14 08:07:45.428148 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.428100    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="fef6e286-c65d-46bd-a85f-aa6eca82dbaa" containerName="istio-proxy"
Sep 14 08:07:45.428148 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.428137    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0d3cbc9-be52-4983-901e-76e256c6abae" containerName="istio-proxy"
Sep 14 08:07:45.428148 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.428145    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0d3cbc9-be52-4983-901e-76e256c6abae" containerName="istio-proxy"
Sep 14 08:07:45.428395 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.428155    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0d3cbc9-be52-4983-901e-76e256c6abae" containerName="istio-proxy"
Sep 14 08:07:45.428395 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.428165    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0d3cbc9-be52-4983-901e-76e256c6abae" containerName="istio-proxy"
Sep 14 08:07:45.428395 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.428284    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fef6e286-c65d-46bd-a85f-aa6eca82dbaa" containerName="istio-proxy"
Sep 14 08:07:45.428395 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.428298    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0d3cbc9-be52-4983-901e-76e256c6abae" containerName="istio-proxy"
Sep 14 08:07:45.428395 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.428310    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0d3cbc9-be52-4983-901e-76e256c6abae" containerName="istio-proxy"
Sep 14 08:07:45.433114 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.433088    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.435021 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.435000    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-7746e4-openshift-default-dockercfg-8wtbk\""
Sep 14 08:07:45.443967 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.443944    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"]
Sep 14 08:07:45.455432 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.455404    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:45.455603 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.455484    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-istio-data\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.455603 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.455572    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-workload-socket\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.455729 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.455615    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-credential-socket\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.455729 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.455684    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4hrd\" (UniqueName: \"kubernetes.io/projected/10943269-128b-4c50-82d8-e70c44c3a70f-kube-api-access-g4hrd\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.455835 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.455767    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-istio-envoy\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.455835 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.455804    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10943269-128b-4c50-82d8-e70c44c3a70f-istio-token\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.455835 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.455825    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10943269-128b-4c50-82d8-e70c44c3a70f-istiod-ca-cert\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.455962 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.455857    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-workload-certs\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.455962 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.455911    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10943269-128b-4c50-82d8-e70c44c3a70f-istio-podinfo\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.525158 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.525122    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:45.556517 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.556466    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-istio-envoy\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.556722 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.556538    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10943269-128b-4c50-82d8-e70c44c3a70f-istio-token\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.556722 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.556567    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10943269-128b-4c50-82d8-e70c44c3a70f-istiod-ca-cert\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.556722 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.556606    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-workload-certs\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.556722 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.556650    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10943269-128b-4c50-82d8-e70c44c3a70f-istio-podinfo\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.556722 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.556692    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-istio-data\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.556994 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.556728    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-workload-socket\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.556994 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.556766    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-credential-socket\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.556994 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.556823    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4hrd\" (UniqueName: \"kubernetes.io/projected/10943269-128b-4c50-82d8-e70c44c3a70f-kube-api-access-g4hrd\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.557156 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.557070    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-workload-certs\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.557156 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.557123    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-workload-socket\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.557275 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.557251    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-credential-socket\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.557362 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.557344    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10943269-128b-4c50-82d8-e70c44c3a70f-istiod-ca-cert\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.557409 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.557382    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-istio-data\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.559279 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.559245    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10943269-128b-4c50-82d8-e70c44c3a70f-istio-podinfo\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.559410 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.559255    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-istio-envoy\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.564704 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.564678    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10943269-128b-4c50-82d8-e70c44c3a70f-istio-token\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.564812 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.564745    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4hrd\" (UniqueName: \"kubernetes.io/projected/10943269-128b-4c50-82d8-e70c44c3a70f-kube-api-access-g4hrd\") pod \"e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") " pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.746029 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.745993    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:45.914147 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:45.914109    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:46.022053 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:46.021958    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"]
Sep 14 08:07:46.026415 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:07:46.026379    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10943269_128b_4c50_82d8_e70c44c3a70f.slice/crio-36a29923dbd521f2dbc52a20eeb397e25285b8b43499f0995cd0ff04f83969ab WatchSource:0}: Error finding container 36a29923dbd521f2dbc52a20eeb397e25285b8b43499f0995cd0ff04f83969ab: Status 404 returned error can't find the container with id 36a29923dbd521f2dbc52a20eeb397e25285b8b43499f0995cd0ff04f83969ab
Sep 14 08:07:46.028569 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:46.028537    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:07:46.028688 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:46.028602    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:07:46.028688 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:46.028630    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:07:46.968848 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:46.968811    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn" event={"ID":"10943269-128b-4c50-82d8-e70c44c3a70f","Type":"ContainerStarted","Data":"5c8c2c56d5038986301d4735387235c528f2867d0ee1a5f0e5fab15ae8f097c7"}
Sep 14 08:07:46.969218 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:46.968876    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn" event={"ID":"10943269-128b-4c50-82d8-e70c44c3a70f","Type":"ContainerStarted","Data":"36a29923dbd521f2dbc52a20eeb397e25285b8b43499f0995cd0ff04f83969ab"}
Sep 14 08:07:46.987467 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:46.987401    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn" podStartSLOduration=1.987381444 podStartE2EDuration="1.987381444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:46.98700382 +0000 UTC m=+1169.443453813" watchObservedRunningTime="2026-09-14 08:07:46.987381444 +0000 UTC m=+1169.443831438"
Sep 14 08:07:47.746981 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:47.746942    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:47.751755 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:47.751728    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:47.974392 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:47.974355    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:47.975417 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:47.975395    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:07:48.189775 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:48.189745    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:49.321667 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.321637    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:49.421858 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.421821    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:49.832387 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.832361    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:07:49.899341 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.899245    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-credential-socket\") pod \"8512cd08-a696-4695-9bbf-c6c3d68d7329\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") "
Sep 14 08:07:49.899341 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.899287    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-token\") pod \"8512cd08-a696-4695-9bbf-c6c3d68d7329\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") "
Sep 14 08:07:49.899341 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.899316    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-podinfo\") pod \"8512cd08-a696-4695-9bbf-c6c3d68d7329\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") "
Sep 14 08:07:49.899670 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.899377    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vc26p\" (UniqueName: \"kubernetes.io/projected/8512cd08-a696-4695-9bbf-c6c3d68d7329-kube-api-access-vc26p\") pod \"8512cd08-a696-4695-9bbf-c6c3d68d7329\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") "
Sep 14 08:07:49.899670 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.899485    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-workload-socket\") pod \"8512cd08-a696-4695-9bbf-c6c3d68d7329\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") "
Sep 14 08:07:49.899670 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.899607    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-envoy\") pod \"8512cd08-a696-4695-9bbf-c6c3d68d7329\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") "
Sep 14 08:07:49.899670 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.899629    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8512cd08-a696-4695-9bbf-c6c3d68d7329" (UID: "8512cd08-a696-4695-9bbf-c6c3d68d7329"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:49.899670 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.899671    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-data\") pod \"8512cd08-a696-4695-9bbf-c6c3d68d7329\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") "
Sep 14 08:07:49.899940 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.899717    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-workload-certs\") pod \"8512cd08-a696-4695-9bbf-c6c3d68d7329\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") "
Sep 14 08:07:49.899940 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.899753    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8512cd08-a696-4695-9bbf-c6c3d68d7329-istiod-ca-cert\") pod \"8512cd08-a696-4695-9bbf-c6c3d68d7329\" (UID: \"8512cd08-a696-4695-9bbf-c6c3d68d7329\") "
Sep 14 08:07:49.900046 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.899950    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-data" (OuterVolumeSpecName: "istio-data") pod "8512cd08-a696-4695-9bbf-c6c3d68d7329" (UID: "8512cd08-a696-4695-9bbf-c6c3d68d7329"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:49.900167 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.900146    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8512cd08-a696-4695-9bbf-c6c3d68d7329" (UID: "8512cd08-a696-4695-9bbf-c6c3d68d7329"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:49.900231 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.900192    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8512cd08-a696-4695-9bbf-c6c3d68d7329" (UID: "8512cd08-a696-4695-9bbf-c6c3d68d7329"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:49.900286 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.900239    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8512cd08-a696-4695-9bbf-c6c3d68d7329-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8512cd08-a696-4695-9bbf-c6c3d68d7329" (UID: "8512cd08-a696-4695-9bbf-c6c3d68d7329"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:49.900345 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.900296    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-data\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:49.900345 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.900314    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-workload-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:49.900345 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.900328    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-credential-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:49.900345 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.900341    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-workload-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:49.901702 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.901671    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-token" (OuterVolumeSpecName: "istio-token") pod "8512cd08-a696-4695-9bbf-c6c3d68d7329" (UID: "8512cd08-a696-4695-9bbf-c6c3d68d7329"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:49.901834 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.901714    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8512cd08-a696-4695-9bbf-c6c3d68d7329-kube-api-access-vc26p" (OuterVolumeSpecName: "kube-api-access-vc26p") pod "8512cd08-a696-4695-9bbf-c6c3d68d7329" (UID: "8512cd08-a696-4695-9bbf-c6c3d68d7329"). InnerVolumeSpecName "kube-api-access-vc26p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:49.901834 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.901816    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8512cd08-a696-4695-9bbf-c6c3d68d7329" (UID: "8512cd08-a696-4695-9bbf-c6c3d68d7329"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:07:49.902178 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.902154    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8512cd08-a696-4695-9bbf-c6c3d68d7329" (UID: "8512cd08-a696-4695-9bbf-c6c3d68d7329"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:49.992842 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.992802    2562 generic.go:356] "Generic (PLEG): container finished" podID="8512cd08-a696-4695-9bbf-c6c3d68d7329" containerID="d17232d3dd3ad19f8744ad693a56d330898dabec149cd26fd250c1621a12f15b" exitCode=0
Sep 14 08:07:49.993019 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.992859    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"
Sep 14 08:07:49.993019 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.992884    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt" event={"ID":"8512cd08-a696-4695-9bbf-c6c3d68d7329","Type":"ContainerDied","Data":"d17232d3dd3ad19f8744ad693a56d330898dabec149cd26fd250c1621a12f15b"}
Sep 14 08:07:49.993019 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.992961    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt" event={"ID":"8512cd08-a696-4695-9bbf-c6c3d68d7329","Type":"ContainerDied","Data":"948433736fbcc19dc41fdeb1a778c3131008a9e08f74b229953ca47a04a86df2"}
Sep 14 08:07:49.993651 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:49.992984    2562 scope.go:117] "RemoveContainer" containerID="d17232d3dd3ad19f8744ad693a56d330898dabec149cd26fd250c1621a12f15b"
Sep 14 08:07:50.000896 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:50.000875    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vc26p\" (UniqueName: \"kubernetes.io/projected/8512cd08-a696-4695-9bbf-c6c3d68d7329-kube-api-access-vc26p\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:50.000998 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:50.000900    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-envoy\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:50.000998 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:50.000914    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8512cd08-a696-4695-9bbf-c6c3d68d7329-istiod-ca-cert\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:50.000998 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:50.000925    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-token\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:50.000998 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:50.000936    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8512cd08-a696-4695-9bbf-c6c3d68d7329-istio-podinfo\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:07:50.003176 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:50.003157    2562 scope.go:117] "RemoveContainer" containerID="891e088b638cc13f7d54787c978c0b4a218e3fd2e2ce6c3df589a7e2ad74c0b9"
Sep 14 08:07:50.011224 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:50.011200    2562 scope.go:117] "RemoveContainer" containerID="d17232d3dd3ad19f8744ad693a56d330898dabec149cd26fd250c1621a12f15b"
Sep 14 08:07:50.011574 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:07:50.011548    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d17232d3dd3ad19f8744ad693a56d330898dabec149cd26fd250c1621a12f15b\": container with ID starting with d17232d3dd3ad19f8744ad693a56d330898dabec149cd26fd250c1621a12f15b not found: ID does not exist" containerID="d17232d3dd3ad19f8744ad693a56d330898dabec149cd26fd250c1621a12f15b"
Sep 14 08:07:50.011728 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:50.011584    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d17232d3dd3ad19f8744ad693a56d330898dabec149cd26fd250c1621a12f15b"} err="failed to get container status \"d17232d3dd3ad19f8744ad693a56d330898dabec149cd26fd250c1621a12f15b\": rpc error: code = NotFound desc = could not find container \"d17232d3dd3ad19f8744ad693a56d330898dabec149cd26fd250c1621a12f15b\": container with ID starting with d17232d3dd3ad19f8744ad693a56d330898dabec149cd26fd250c1621a12f15b not found: ID does not exist"
Sep 14 08:07:50.011728 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:50.011608    2562 scope.go:117] "RemoveContainer" containerID="891e088b638cc13f7d54787c978c0b4a218e3fd2e2ce6c3df589a7e2ad74c0b9"
Sep 14 08:07:50.012052 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:07:50.012025    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"891e088b638cc13f7d54787c978c0b4a218e3fd2e2ce6c3df589a7e2ad74c0b9\": container with ID starting with 891e088b638cc13f7d54787c978c0b4a218e3fd2e2ce6c3df589a7e2ad74c0b9 not found: ID does not exist" containerID="891e088b638cc13f7d54787c978c0b4a218e3fd2e2ce6c3df589a7e2ad74c0b9"
Sep 14 08:07:50.012138 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:50.012055    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"891e088b638cc13f7d54787c978c0b4a218e3fd2e2ce6c3df589a7e2ad74c0b9"} err="failed to get container status \"891e088b638cc13f7d54787c978c0b4a218e3fd2e2ce6c3df589a7e2ad74c0b9\": rpc error: code = NotFound desc = could not find container \"891e088b638cc13f7d54787c978c0b4a218e3fd2e2ce6c3df589a7e2ad74c0b9\": container with ID starting with 891e088b638cc13f7d54787c978c0b4a218e3fd2e2ce6c3df589a7e2ad74c0b9 not found: ID does not exist"
Sep 14 08:07:50.013927 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:50.013906    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"]
Sep 14 08:07:50.019539 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:50.019517    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-5ae4e6-openshift-default-6b75f7c9b8-bg8tt"]
Sep 14 08:07:50.087643 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:50.087610    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:50.112429 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:50.112393    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8512cd08-a696-4695-9bbf-c6c3d68d7329" path="/var/lib/kubelet/pods/8512cd08-a696-4695-9bbf-c6c3d68d7329/volumes"
Sep 14 08:07:51.049065 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:51.049031    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:51.757335 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:51.757296    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:52.582993 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:52.582957    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:52.683550 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:52.683515    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:53.019221 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:53.019187    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:53.138962 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:53.138924    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:07:53.225228 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:07:53.225191    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:02.113240 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:02.113208    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:08:04.087678 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.087644    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"]
Sep 14 08:08:04.088059 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.088037    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8512cd08-a696-4695-9bbf-c6c3d68d7329" containerName="istio-proxy"
Sep 14 08:08:04.088059 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.088050    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8512cd08-a696-4695-9bbf-c6c3d68d7329" containerName="istio-proxy"
Sep 14 08:08:04.088059 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.088059    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8512cd08-a696-4695-9bbf-c6c3d68d7329" containerName="istio-proxy"
Sep 14 08:08:04.088160 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.088064    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8512cd08-a696-4695-9bbf-c6c3d68d7329" containerName="istio-proxy"
Sep 14 08:08:04.088160 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.088127    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8512cd08-a696-4695-9bbf-c6c3d68d7329" containerName="istio-proxy"
Sep 14 08:08:04.088160 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.088134    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="fef6e286-c65d-46bd-a85f-aa6eca82dbaa" containerName="istio-proxy"
Sep 14 08:08:04.092699 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.092677    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:04.094486 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.094459    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b69a34\"/\"e2e-conflict-b-cad94a07-kserve-self-signed-certs\""
Sep 14 08:08:04.100048 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.099650    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"]
Sep 14 08:08:04.249235 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.249166    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-home\") pod \"e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:04.249466 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.249248    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-model-cache\") pod \"e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:04.249466 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.249279    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-tmp-dir\") pod \"e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:04.249466 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.249315    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-dshm\") pod \"e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:04.249466 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.249360    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gshvl\" (UniqueName: \"kubernetes.io/projected/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-kube-api-access-gshvl\") pod \"e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:04.249466 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.249427    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-tls-certs\") pod \"e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:04.350337 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.350229    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-home\") pod \"e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:04.350637 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.350613    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-model-cache\") pod \"e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:04.350765 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.350747    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-tmp-dir\") pod \"e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:04.350885 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.350870    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-dshm\") pod \"e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:04.351039 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.351020    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gshvl\" (UniqueName: \"kubernetes.io/projected/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-kube-api-access-gshvl\") pod \"e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:04.351204 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.351189    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-tls-certs\") pod \"e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:04.352277 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.352249    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-tmp-dir\") pod \"e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:04.352453 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.352269    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-home\") pod \"e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:04.352580 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.352486    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-model-cache\") pod \"e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:04.355033 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.354985    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-dshm\") pod \"e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:04.355670 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.355632    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-tls-certs\") pod \"e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:04.369552 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.367579    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gshvl\" (UniqueName: \"kubernetes.io/projected/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-kube-api-access-gshvl\") pod \"e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:04.388789 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.388753    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:04.402642 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.402611    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:04.709346 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:04.709311    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"]
Sep 14 08:08:04.712870 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:08:04.712845    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ac18e99_fc19_4a4f_9f66_6cd4893ceae0.slice/crio-5e0025bd4b6bf20830cc19f9ea194f7529c61eef22618f9cc5c2f6ccde575a0e WatchSource:0}: Error finding container 5e0025bd4b6bf20830cc19f9ea194f7529c61eef22618f9cc5c2f6ccde575a0e: Status 404 returned error can't find the container with id 5e0025bd4b6bf20830cc19f9ea194f7529c61eef22618f9cc5c2f6ccde575a0e
Sep 14 08:08:05.079070 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:05.078972    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h" event={"ID":"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0","Type":"ContainerStarted","Data":"4aac3c4bdaf8728346d689b197dc9b5fe7e096e5b3cd662d7545c86efe69bafa"}
Sep 14 08:08:05.079070 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:05.079015    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h" event={"ID":"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0","Type":"ContainerStarted","Data":"5e0025bd4b6bf20830cc19f9ea194f7529c61eef22618f9cc5c2f6ccde575a0e"}
Sep 14 08:08:05.098429 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:05.098361    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h" podStartSLOduration=1.098342256 podStartE2EDuration="1.098342256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:08:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:08:05.097907715 +0000 UTC m=+1187.554357709" watchObservedRunningTime="2026-09-14 08:08:05.098342256 +0000 UTC m=+1187.554792249"
Sep 14 08:08:13.122695 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.122661    2562 generic.go:356] "Generic (PLEG): container finished" podID="10943269-128b-4c50-82d8-e70c44c3a70f" containerID="5c8c2c56d5038986301d4735387235c528f2867d0ee1a5f0e5fab15ae8f097c7" exitCode=0
Sep 14 08:08:13.123117 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.122741    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn" event={"ID":"10943269-128b-4c50-82d8-e70c44c3a70f","Type":"ContainerDied","Data":"5c8c2c56d5038986301d4735387235c528f2867d0ee1a5f0e5fab15ae8f097c7"}
Sep 14 08:08:13.123185 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.123158    2562 scope.go:117] "RemoveContainer" containerID="5c8c2c56d5038986301d4735387235c528f2867d0ee1a5f0e5fab15ae8f097c7"
Sep 14 08:08:13.124373 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.124342    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:08:13.124484 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.124427    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:08:13.124569 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.124495    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:08:13.284661 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.284627    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"]
Sep 14 08:08:13.285147 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.285129    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8512cd08-a696-4695-9bbf-c6c3d68d7329" containerName="istio-proxy"
Sep 14 08:08:13.288176 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.288154    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:13.290184 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.290159    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-48e079\"/\"kube-root-ca.crt\""
Sep 14 08:08:13.290347 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.290327    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-48e079\"/\"openshift-service-ca.crt\""
Sep 14 08:08:13.290441 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.290408    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-48e079\"/\"default-dockercfg-2sqwq\""
Sep 14 08:08:13.290542 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.290494    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-48e079\"/\"noten-model-48e079-kserve-self-signed-certs\""
Sep 14 08:08:13.296322 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.296296    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"]
Sep 14 08:08:13.397656 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.397563    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:13.455964 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.455924    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-model-cache\") pod \"noten-model-48e079-kserve-7db4dd677f-6mcss\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") " pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:13.456144 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.455985    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-tmp-dir\") pod \"noten-model-48e079-kserve-7db4dd677f-6mcss\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") " pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:13.456144 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.456113    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vssr\" (UniqueName: \"kubernetes.io/projected/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-kube-api-access-2vssr\") pod \"noten-model-48e079-kserve-7db4dd677f-6mcss\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") " pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:13.456249 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.456157    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-home\") pod \"noten-model-48e079-kserve-7db4dd677f-6mcss\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") " pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:13.456249 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.456232    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-tls-certs\") pod \"noten-model-48e079-kserve-7db4dd677f-6mcss\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") " pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:13.456361 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.456319    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-dshm\") pod \"noten-model-48e079-kserve-7db4dd677f-6mcss\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") " pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:13.556886 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.556842    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-tmp-dir\") pod \"noten-model-48e079-kserve-7db4dd677f-6mcss\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") " pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:13.557083 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.556904    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vssr\" (UniqueName: \"kubernetes.io/projected/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-kube-api-access-2vssr\") pod \"noten-model-48e079-kserve-7db4dd677f-6mcss\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") " pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:13.557083 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.556928    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-home\") pod \"noten-model-48e079-kserve-7db4dd677f-6mcss\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") " pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:13.557083 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.556967    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-tls-certs\") pod \"noten-model-48e079-kserve-7db4dd677f-6mcss\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") " pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:13.557263 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.557110    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-dshm\") pod \"noten-model-48e079-kserve-7db4dd677f-6mcss\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") " pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:13.557263 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.557210    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-model-cache\") pod \"noten-model-48e079-kserve-7db4dd677f-6mcss\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") " pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:13.557263 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.557254    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-tmp-dir\") pod \"noten-model-48e079-kserve-7db4dd677f-6mcss\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") " pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:13.557413 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.557278    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-home\") pod \"noten-model-48e079-kserve-7db4dd677f-6mcss\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") " pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:13.557520 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.557486    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-model-cache\") pod \"noten-model-48e079-kserve-7db4dd677f-6mcss\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") " pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:13.559281 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.559250    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-dshm\") pod \"noten-model-48e079-kserve-7db4dd677f-6mcss\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") " pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:13.559584 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.559565    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-tls-certs\") pod \"noten-model-48e079-kserve-7db4dd677f-6mcss\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") " pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:13.564831 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.564810    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vssr\" (UniqueName: \"kubernetes.io/projected/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-kube-api-access-2vssr\") pod \"noten-model-48e079-kserve-7db4dd677f-6mcss\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") " pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:13.598839 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.598793    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:13.886585 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:13.886555    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"]
Sep 14 08:08:13.888239 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:08:13.888212    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podabfa0bd1_0e2e_499c_9c1e_8ede2fc62e4f.slice/crio-23fa02aa2a2cc2938bd9f423bda7b53d608f7fcce144abe8d86918b009f5d713 WatchSource:0}: Error finding container 23fa02aa2a2cc2938bd9f423bda7b53d608f7fcce144abe8d86918b009f5d713: Status 404 returned error can't find the container with id 23fa02aa2a2cc2938bd9f423bda7b53d608f7fcce144abe8d86918b009f5d713
Sep 14 08:08:14.129330 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:14.129296    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn" event={"ID":"10943269-128b-4c50-82d8-e70c44c3a70f","Type":"ContainerStarted","Data":"a9efb7002bb14365936ada3400b796af886a107ad80cd1c391d590243203eaca"}
Sep 14 08:08:14.130918 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:14.130892    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss" event={"ID":"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f","Type":"ContainerStarted","Data":"1906c5a21027b84b489323df02a486796024f5d8bd4040919978fc97faedc8fc"}
Sep 14 08:08:14.131052 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:14.130926    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss" event={"ID":"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f","Type":"ContainerStarted","Data":"23fa02aa2a2cc2938bd9f423bda7b53d608f7fcce144abe8d86918b009f5d713"}
Sep 14 08:08:14.167719 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:14.167606    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss" podStartSLOduration=1.167584435 podStartE2EDuration="1.167584435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:08:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:08:14.166061143 +0000 UTC m=+1196.622511137" watchObservedRunningTime="2026-09-14 08:08:14.167584435 +0000 UTC m=+1196.624034429"
Sep 14 08:08:14.402952 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:14.402915    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:14.403228 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:14.403202    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:14.415772 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:14.415740    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:14.726922 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:14.726889    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:14.746692 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:14.746648    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:08:14.748094 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:14.748058    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 14 08:08:14.748244 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:14.748140    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn" podUID="10943269-128b-4c50-82d8-e70c44c3a70f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 14 08:08:15.171753 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:15.171659    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:08:15.500971 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:15.500932    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:08:15.521590 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:15.521555    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:15.747142 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:15.747101    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 14 08:08:15.747451 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:15.747408    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn" podUID="10943269-128b-4c50-82d8-e70c44c3a70f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 14 08:08:16.345379 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:16.345325    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:16.747572 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:16.747525    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 14 08:08:16.747760 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:16.747615    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn" podUID="10943269-128b-4c50-82d8-e70c44c3a70f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 14 08:08:17.115450 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:17.115353    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:17.746948 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:17.746896    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 14 08:08:17.747484 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:17.746990    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn" podUID="10943269-128b-4c50-82d8-e70c44c3a70f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 14 08:08:17.885245 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:17.885214    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:17.974319 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:17.974278    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:08:18.062283 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:18.062196    2562 scope.go:117] "RemoveContainer" containerID="35b292d156d9a78416af31c91cc99490c87c23f0cc729d49a1ea53a79e3e06ce"
Sep 14 08:08:18.651633 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:18.651593    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:18.746961 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:18.746921    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 14 08:08:18.747613 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:18.747577    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn" podUID="10943269-128b-4c50-82d8-e70c44c3a70f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 14 08:08:18.958263 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:18.958213    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:19.289271 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.289167    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:19.558794 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.558699    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"]
Sep 14 08:08:19.565188 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.565158    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.567648 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.567621    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-ea9d35-openshift-default-dockercfg-kb496\""
Sep 14 08:08:19.577002 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.576452    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"]
Sep 14 08:08:19.594480 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.594441    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:19.620654 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.620616    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-credential-socket\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.620830 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.620661    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7kss\" (UniqueName: \"kubernetes.io/projected/c58ec321-5017-43e7-adbe-107cd0f3b7c4-kube-api-access-w7kss\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.620830 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.620783    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-workload-certs\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.620927 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.620832    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istiod-ca-cert\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.620927 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.620852    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-data\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.620927 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.620903    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-podinfo\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.621040 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.620931    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-workload-socket\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.621040 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.620983    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-token\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.621040 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.621020    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-envoy\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.721740 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.721693    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-workload-certs\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.721930 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.721763    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istiod-ca-cert\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.721930 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.721792    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-data\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.721930 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.721832    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-podinfo\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.721930 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.721871    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-workload-socket\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.722226 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.721934    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-token\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.722226 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.721988    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-envoy\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.722226 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.722017    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-credential-socket\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.722226 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.722046    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7kss\" (UniqueName: \"kubernetes.io/projected/c58ec321-5017-43e7-adbe-107cd0f3b7c4-kube-api-access-w7kss\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.722226 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.722168    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-data\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.722484 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.722286    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-workload-socket\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.722484 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.722373    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-workload-certs\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.722484 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.722379    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-credential-socket\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.722755 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.722727    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istiod-ca-cert\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.724421 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.724395    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-envoy\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.724737 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.724717    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-podinfo\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.730163 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.730143    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-token\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.730459 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.730436    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7kss\" (UniqueName: \"kubernetes.io/projected/c58ec321-5017-43e7-adbe-107cd0f3b7c4-kube-api-access-w7kss\") pod \"e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") " pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:19.751205 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.751181    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:08:19.752310 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.752289    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:08:19.876712 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.876617    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:19.881491 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:19.881457    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:20.169469 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:20.169435    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"]
Sep 14 08:08:20.171168 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:08:20.171139    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc58ec321_5017_43e7_adbe_107cd0f3b7c4.slice/crio-66405745b9d0d910a6ea5662c0a771d4dacb5ba76930d594a4222a54a484e062 WatchSource:0}: Error finding container 66405745b9d0d910a6ea5662c0a771d4dacb5ba76930d594a4222a54a484e062: Status 404 returned error can't find the container with id 66405745b9d0d910a6ea5662c0a771d4dacb5ba76930d594a4222a54a484e062
Sep 14 08:08:20.173684 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:20.173651    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:08:20.173778 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:20.173715    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:08:20.173778 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:20.173744    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:08:20.178884 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:20.178777    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf" event={"ID":"c58ec321-5017-43e7-adbe-107cd0f3b7c4","Type":"ContainerStarted","Data":"66405745b9d0d910a6ea5662c0a771d4dacb5ba76930d594a4222a54a484e062"}
Sep 14 08:08:21.185776 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:21.185741    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf" event={"ID":"c58ec321-5017-43e7-adbe-107cd0f3b7c4","Type":"ContainerStarted","Data":"c05c73a8b9ad9c1f4834e36e6e0904191a2bd203d260985fd36d5de58faeeb28"}
Sep 14 08:08:21.196642 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:21.196614    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:21.205885 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:21.205834    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf" podStartSLOduration=2.205820281 podStartE2EDuration="2.205820281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:08:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:08:21.203021973 +0000 UTC m=+1203.659471967" watchObservedRunningTime="2026-09-14 08:08:21.205820281 +0000 UTC m=+1203.662270273"
Sep 14 08:08:21.298342 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:21.298297    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:21.881680 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:21.881639    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:21.886842 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:21.886810    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:22.190829 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:22.190799    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:22.191725 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:22.191704    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:22.348861 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:22.348819    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:23.444030 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:23.443989    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:23.501861 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:23.501823    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:23.599343 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:23.599301    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:23.599573 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:23.599359    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:23.611484 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:23.611460    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:24.117727 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:24.117690    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:24.211453 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:24.211417    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:25.038311 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:25.038271    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:25.786161 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:25.786118    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:25.995180 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:25.995142    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"]
Sep 14 08:08:25.998960 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:25.998939    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:26.000924 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.000899    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-8b87debd\"/\"openshift-service-ca.crt\""
Sep 14 08:08:26.001031 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.000899    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-8b87debd\"/\"kube-root-ca.crt\""
Sep 14 08:08:26.001380 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.001360    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-8b87debd\"/\"e2e-lifecycle-model-8b87debd-kserve-self-signed-certs\""
Sep 14 08:08:26.001380 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.001373    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-8b87debd\"/\"default-dockercfg-xbf7k\""
Sep 14 08:08:26.005727 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.005706    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"]
Sep 14 08:08:26.089104 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.089003    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-tmp-dir\") pod \"e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") " pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:26.089104 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.089046    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-dshm\") pod \"e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") " pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:26.089104 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.089084    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d0a01fd-3657-4149-a176-9ee4937cb166-tls-certs\") pod \"e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") " pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:26.089726 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.089184    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-home\") pod \"e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") " pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:26.089726 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.089225    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2ngb\" (UniqueName: \"kubernetes.io/projected/6d0a01fd-3657-4149-a176-9ee4937cb166-kube-api-access-k2ngb\") pod \"e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") " pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:26.089726 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.089256    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-model-cache\") pod \"e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") " pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:26.095921 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.095877    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:26.190165 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.190127    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d0a01fd-3657-4149-a176-9ee4937cb166-tls-certs\") pod \"e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") " pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:26.190370 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.190264    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-home\") pod \"e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") " pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:26.190370 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.190285    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2ngb\" (UniqueName: \"kubernetes.io/projected/6d0a01fd-3657-4149-a176-9ee4937cb166-kube-api-access-k2ngb\") pod \"e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") " pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:26.190370 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.190311    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-model-cache\") pod \"e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") " pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:26.190591 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.190460    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-tmp-dir\") pod \"e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") " pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:26.190591 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.190533    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-dshm\") pod \"e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") " pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:26.190754 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.190730    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-home\") pod \"e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") " pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:26.190894 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.190863    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-model-cache\") pod \"e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") " pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:26.191014 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.190986    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-tmp-dir\") pod \"e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") " pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:26.192723 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.192691    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-dshm\") pod \"e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") " pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:26.192825 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.192774    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d0a01fd-3657-4149-a176-9ee4937cb166-tls-certs\") pod \"e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") " pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:26.199953 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.199930    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2ngb\" (UniqueName: \"kubernetes.io/projected/6d0a01fd-3657-4149-a176-9ee4937cb166-kube-api-access-k2ngb\") pod \"e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") " pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:26.310102 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.310067    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:26.585954 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.585924    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"]
Sep 14 08:08:26.587647 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:08:26.587585    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d0a01fd_3657_4149_a176_9ee4937cb166.slice/crio-6493eb4e32ea7ed3e797932cb34406acfdf546d6e1d748efd058e010fffc4df0 WatchSource:0}: Error finding container 6493eb4e32ea7ed3e797932cb34406acfdf546d6e1d748efd058e010fffc4df0: Status 404 returned error can't find the container with id 6493eb4e32ea7ed3e797932cb34406acfdf546d6e1d748efd058e010fffc4df0
Sep 14 08:08:26.651974 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:26.651941    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:27.217375 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:27.217340    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b" event={"ID":"6d0a01fd-3657-4149-a176-9ee4937cb166","Type":"ContainerStarted","Data":"e98b5790fbd20cdf0ebb08ffc799aba630e9bb718e4777e06dc1dc83acc7a7ee"}
Sep 14 08:08:27.217768 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:27.217388    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b" event={"ID":"6d0a01fd-3657-4149-a176-9ee4937cb166","Type":"ContainerStarted","Data":"6493eb4e32ea7ed3e797932cb34406acfdf546d6e1d748efd058e010fffc4df0"}
Sep 14 08:08:27.234849 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:27.234800    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b" podStartSLOduration=2.234785686 podStartE2EDuration="2.234785686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:08:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:08:27.23257321 +0000 UTC m=+1209.689023199" watchObservedRunningTime="2026-09-14 08:08:27.234785686 +0000 UTC m=+1209.691235678"
Sep 14 08:08:27.532783 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:27.532687    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:27.636399 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:27.636361    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:27.821436 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:27.821334    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:28.030784 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:28.030745    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:28.130989 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:28.130898    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:31.182658 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:31.182617    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:36.310598 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:36.310555    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:36.310598 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:36.310604    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:36.323370 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:36.323345    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:37.284448 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:37.284417    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:08:38.971897 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:38.971861    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"]
Sep 14 08:08:39.007342 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:39.007303    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:39.062979 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:39.062936    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"]
Sep 14 08:08:39.063170 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:39.063139    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd" podUID="056170e3-8e2a-450f-936d-29fd341e1d79" containerName="istio-proxy" containerID="cri-o://04573897bfda9ddc72d0454d1163c66da9453254a2f4a8efaba1685d006998e2" gracePeriod=30
Sep 14 08:08:39.105329 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:39.105295    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:39.298335 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:39.298240    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:39.412816 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:39.412781    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:08:39.420790 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:39.420760    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:41.855514 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:41.855454    2562 patch_prober.go:28] interesting pod/e2e-noten-48e079-openshift-default-5989d455cd-gzljd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 08:08:41.856112 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:41.856075    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd" podUID="056170e3-8e2a-450f-936d-29fd341e1d79" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 08:08:44.312321 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.312286    2562 generic.go:356] "Generic (PLEG): container finished" podID="056170e3-8e2a-450f-936d-29fd341e1d79" containerID="04573897bfda9ddc72d0454d1163c66da9453254a2f4a8efaba1685d006998e2" exitCode=0
Sep 14 08:08:44.312741 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.312367    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd" event={"ID":"056170e3-8e2a-450f-936d-29fd341e1d79","Type":"ContainerDied","Data":"04573897bfda9ddc72d0454d1163c66da9453254a2f4a8efaba1685d006998e2"}
Sep 14 08:08:44.364280 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.363977    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:08:44.436326 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.436286    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:44.488186 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.488146    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldkbm\" (UniqueName: \"kubernetes.io/projected/056170e3-8e2a-450f-936d-29fd341e1d79-kube-api-access-ldkbm\") pod \"056170e3-8e2a-450f-936d-29fd341e1d79\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") "
Sep 14 08:08:44.488352 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.488224    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-workload-socket\") pod \"056170e3-8e2a-450f-936d-29fd341e1d79\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") "
Sep 14 08:08:44.488352 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.488275    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-credential-socket\") pod \"056170e3-8e2a-450f-936d-29fd341e1d79\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") "
Sep 14 08:08:44.488352 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.488312    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-workload-certs\") pod \"056170e3-8e2a-450f-936d-29fd341e1d79\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") "
Sep 14 08:08:44.488352 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.488337    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/056170e3-8e2a-450f-936d-29fd341e1d79-istiod-ca-cert\") pod \"056170e3-8e2a-450f-936d-29fd341e1d79\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") "
Sep 14 08:08:44.488491 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.488375    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/056170e3-8e2a-450f-936d-29fd341e1d79-istio-token\") pod \"056170e3-8e2a-450f-936d-29fd341e1d79\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") "
Sep 14 08:08:44.488491 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.488423    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-istio-envoy\") pod \"056170e3-8e2a-450f-936d-29fd341e1d79\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") "
Sep 14 08:08:44.488491 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.488446    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-istio-data\") pod \"056170e3-8e2a-450f-936d-29fd341e1d79\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") "
Sep 14 08:08:44.488663 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.488491    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/056170e3-8e2a-450f-936d-29fd341e1d79-istio-podinfo\") pod \"056170e3-8e2a-450f-936d-29fd341e1d79\" (UID: \"056170e3-8e2a-450f-936d-29fd341e1d79\") "
Sep 14 08:08:44.488663 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.488544    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "056170e3-8e2a-450f-936d-29fd341e1d79" (UID: "056170e3-8e2a-450f-936d-29fd341e1d79"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:08:44.488663 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.488603    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "056170e3-8e2a-450f-936d-29fd341e1d79" (UID: "056170e3-8e2a-450f-936d-29fd341e1d79"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:08:44.488663 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.488614    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "056170e3-8e2a-450f-936d-29fd341e1d79" (UID: "056170e3-8e2a-450f-936d-29fd341e1d79"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:08:44.488858 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.488778    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-istio-data" (OuterVolumeSpecName: "istio-data") pod "056170e3-8e2a-450f-936d-29fd341e1d79" (UID: "056170e3-8e2a-450f-936d-29fd341e1d79"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:08:44.488922 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.488857    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/056170e3-8e2a-450f-936d-29fd341e1d79-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "056170e3-8e2a-450f-936d-29fd341e1d79" (UID: "056170e3-8e2a-450f-936d-29fd341e1d79"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:08:44.488977 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.488968    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-workload-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:08:44.489025 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.488981    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-credential-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:08:44.489025 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.488990    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-workload-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:08:44.489025 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.488998    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/056170e3-8e2a-450f-936d-29fd341e1d79-istiod-ca-cert\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:08:44.489025 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.489006    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-istio-data\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:08:44.490519 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.490474    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/056170e3-8e2a-450f-936d-29fd341e1d79-kube-api-access-ldkbm" (OuterVolumeSpecName: "kube-api-access-ldkbm") pod "056170e3-8e2a-450f-936d-29fd341e1d79" (UID: "056170e3-8e2a-450f-936d-29fd341e1d79"). InnerVolumeSpecName "kube-api-access-ldkbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:08:44.490716 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.490691    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "056170e3-8e2a-450f-936d-29fd341e1d79" (UID: "056170e3-8e2a-450f-936d-29fd341e1d79"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:08:44.490771 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.490695    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/056170e3-8e2a-450f-936d-29fd341e1d79-istio-token" (OuterVolumeSpecName: "istio-token") pod "056170e3-8e2a-450f-936d-29fd341e1d79" (UID: "056170e3-8e2a-450f-936d-29fd341e1d79"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:08:44.490907 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.490889    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/056170e3-8e2a-450f-936d-29fd341e1d79-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "056170e3-8e2a-450f-936d-29fd341e1d79" (UID: "056170e3-8e2a-450f-936d-29fd341e1d79"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:08:44.590561 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.590436    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/056170e3-8e2a-450f-936d-29fd341e1d79-istio-podinfo\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:08:44.590561 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.590474    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldkbm\" (UniqueName: \"kubernetes.io/projected/056170e3-8e2a-450f-936d-29fd341e1d79-kube-api-access-ldkbm\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:08:44.590561 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.590484    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/056170e3-8e2a-450f-936d-29fd341e1d79-istio-token\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:08:44.590561 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:44.590492    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/056170e3-8e2a-450f-936d-29fd341e1d79-istio-envoy\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:08:45.319189 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:45.319156    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"
Sep 14 08:08:45.319189 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:45.319175    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd" event={"ID":"056170e3-8e2a-450f-936d-29fd341e1d79","Type":"ContainerDied","Data":"6730d061620717971f67520968cf33acb8b456dc958d473c3cca7024ca0562d4"}
Sep 14 08:08:45.319722 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:45.319239    2562 scope.go:117] "RemoveContainer" containerID="04573897bfda9ddc72d0454d1163c66da9453254a2f4a8efaba1685d006998e2"
Sep 14 08:08:45.350525 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:45.349137    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"]
Sep 14 08:08:45.361527 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:45.359385    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-48e079-openshift-default-5989d455cd-gzljd"]
Sep 14 08:08:46.113431 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:46.113397    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="056170e3-8e2a-450f-936d-29fd341e1d79" path="/var/lib/kubelet/pods/056170e3-8e2a-450f-936d-29fd341e1d79/volumes"
Sep 14 08:08:49.692181 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:49.692134    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:50.349082 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:50.349043    2562 generic.go:356] "Generic (PLEG): container finished" podID="c58ec321-5017-43e7-adbe-107cd0f3b7c4" containerID="c05c73a8b9ad9c1f4834e36e6e0904191a2bd203d260985fd36d5de58faeeb28" exitCode=0
Sep 14 08:08:50.349256 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:50.349112    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf" event={"ID":"c58ec321-5017-43e7-adbe-107cd0f3b7c4","Type":"ContainerDied","Data":"c05c73a8b9ad9c1f4834e36e6e0904191a2bd203d260985fd36d5de58faeeb28"}
Sep 14 08:08:50.349549 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:50.349532    2562 scope.go:117] "RemoveContainer" containerID="c05c73a8b9ad9c1f4834e36e6e0904191a2bd203d260985fd36d5de58faeeb28"
Sep 14 08:08:50.350783 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:50.350757    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:08:50.350897 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:50.350822    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:08:50.350897 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:50.350856    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:08:51.360993 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:51.360942    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf" event={"ID":"c58ec321-5017-43e7-adbe-107cd0f3b7c4","Type":"ContainerStarted","Data":"2bd6ebe2b104f8dbaafd0eaf8b7d06c8fe5c2867a61c4b2b8b0dce89d9e01051"}
Sep 14 08:08:51.777716 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:51.777682    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"]
Sep 14 08:08:51.882048 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:51.881994    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:51.883609 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:51.883577    2562 patch_prober.go:28] interesting pod/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 14 08:08:51.883751 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:51.883639    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf" podUID="c58ec321-5017-43e7-adbe-107cd0f3b7c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 14 08:08:51.958230 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:51.958191    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:52.191115 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:52.191088    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:52.192331 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:52.192302    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:52.882082 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:52.882039    2562 patch_prober.go:28] interesting pod/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 14 08:08:52.882476 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:52.882100    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf" podUID="c58ec321-5017-43e7-adbe-107cd0f3b7c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 14 08:08:53.038330 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:53.038291    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:53.136015 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:53.135909    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:53.225192 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:53.225164    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:53.882581 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:53.882528    2562 patch_prober.go:28] interesting pod/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 14 08:08:53.883117 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:53.882611    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf" podUID="c58ec321-5017-43e7-adbe-107cd0f3b7c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 14 08:08:53.914527 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:53.914484    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:53.990680 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:53.990634    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss" podUID="abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f" containerName="main" containerID="cri-o://1906c5a21027b84b489323df02a486796024f5d8bd4040919978fc97faedc8fc" gracePeriod=45
Sep 14 08:08:54.286652 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.286627    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:54.378942 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.378907    2562 generic.go:356] "Generic (PLEG): container finished" podID="abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f" containerID="1906c5a21027b84b489323df02a486796024f5d8bd4040919978fc97faedc8fc" exitCode=0
Sep 14 08:08:54.379147 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.378948    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"
Sep 14 08:08:54.379147 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.378995    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss" event={"ID":"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f","Type":"ContainerDied","Data":"1906c5a21027b84b489323df02a486796024f5d8bd4040919978fc97faedc8fc"}
Sep 14 08:08:54.379147 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.379039    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss" event={"ID":"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f","Type":"ContainerDied","Data":"23fa02aa2a2cc2938bd9f423bda7b53d608f7fcce144abe8d86918b009f5d713"}
Sep 14 08:08:54.379147 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.379063    2562 scope.go:117] "RemoveContainer" containerID="1906c5a21027b84b489323df02a486796024f5d8bd4040919978fc97faedc8fc"
Sep 14 08:08:54.387576 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.387555    2562 scope.go:117] "RemoveContainer" containerID="1906c5a21027b84b489323df02a486796024f5d8bd4040919978fc97faedc8fc"
Sep 14 08:08:54.387861 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:08:54.387842    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1906c5a21027b84b489323df02a486796024f5d8bd4040919978fc97faedc8fc\": container with ID starting with 1906c5a21027b84b489323df02a486796024f5d8bd4040919978fc97faedc8fc not found: ID does not exist" containerID="1906c5a21027b84b489323df02a486796024f5d8bd4040919978fc97faedc8fc"
Sep 14 08:08:54.387910 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.387874    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1906c5a21027b84b489323df02a486796024f5d8bd4040919978fc97faedc8fc"} err="failed to get container status \"1906c5a21027b84b489323df02a486796024f5d8bd4040919978fc97faedc8fc\": rpc error: code = NotFound desc = could not find container \"1906c5a21027b84b489323df02a486796024f5d8bd4040919978fc97faedc8fc\": container with ID starting with 1906c5a21027b84b489323df02a486796024f5d8bd4040919978fc97faedc8fc not found: ID does not exist"
Sep 14 08:08:54.391393 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.391363    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-tmp-dir\") pod \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") "
Sep 14 08:08:54.391393 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.391391    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-home\") pod \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") "
Sep 14 08:08:54.391585 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.391459    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-model-cache\") pod \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") "
Sep 14 08:08:54.391585 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.391477    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-tls-certs\") pod \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") "
Sep 14 08:08:54.391585 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.391525    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vssr\" (UniqueName: \"kubernetes.io/projected/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-kube-api-access-2vssr\") pod \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") "
Sep 14 08:08:54.391585 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.391570    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-dshm\") pod \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\" (UID: \"abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f\") "
Sep 14 08:08:54.391795 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.391707    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f" (UID: "abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:08:54.391795 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.391754    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-model-cache" (OuterVolumeSpecName: "model-cache") pod "abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f" (UID: "abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:08:54.391795 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.391783    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-home" (OuterVolumeSpecName: "home") pod "abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f" (UID: "abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:08:54.391987 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.391969    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-tmp-dir\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:08:54.391987 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.391987    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-home\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:08:54.392140 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.392002    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-model-cache\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:08:54.393885 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.393850    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-kube-api-access-2vssr" (OuterVolumeSpecName: "kube-api-access-2vssr") pod "abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f" (UID: "abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f"). InnerVolumeSpecName "kube-api-access-2vssr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:08:54.393885 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.393877    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-dshm" (OuterVolumeSpecName: "dshm") pod "abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f" (UID: "abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:08:54.394011 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.393909    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f" (UID: "abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:08:54.493126 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.493079    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-tls-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:08:54.493126 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.493118    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vssr\" (UniqueName: \"kubernetes.io/projected/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-kube-api-access-2vssr\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:08:54.493126 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.493128    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f-dshm\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:08:54.706903 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.706867    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"]
Sep 14 08:08:54.710316 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.710287    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss"]
Sep 14 08:08:54.825280 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.825185    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:54.882681 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.882641    2562 patch_prober.go:28] interesting pod/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 14 08:08:54.883078 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:54.882724    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf" podUID="c58ec321-5017-43e7-adbe-107cd0f3b7c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 14 08:08:55.882261 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:55.882218    2562 patch_prober.go:28] interesting pod/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Sep 14 08:08:55.882416 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:55.882283    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf" podUID="c58ec321-5017-43e7-adbe-107cd0f3b7c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Sep 14 08:08:56.113454 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:56.113367    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f" path="/var/lib/kubelet/pods/abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f/volumes"
Sep 14 08:08:56.886345 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:56.886312    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:56.887429 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:56.887409    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:08:57.005352 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:57.005317    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:57.106568 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:57.106529    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:08:59.200797 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:59.200743    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-noten-48e079/noten-model-48e079-kserve-7db4dd677f-6mcss" podUID="abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Sep 14 08:08:59.296955 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:08:59.296921    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:09:06.797610 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:06.797571    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h" podUID="6ac18e99-fc19-4a4f-9f66-6cd4893ceae0" containerName="main" containerID="cri-o://4aac3c4bdaf8728346d689b197dc9b5fe7e096e5b3cd662d7545c86efe69bafa" gracePeriod=45
Sep 14 08:09:06.997745 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:06.997706    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"]
Sep 14 08:09:06.998162 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:06.998139    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="056170e3-8e2a-450f-936d-29fd341e1d79" containerName="istio-proxy"
Sep 14 08:09:06.998162 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:06.998157    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="056170e3-8e2a-450f-936d-29fd341e1d79" containerName="istio-proxy"
Sep 14 08:09:06.998424 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:06.998207    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f" containerName="main"
Sep 14 08:09:06.998424 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:06.998214    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f" containerName="main"
Sep 14 08:09:06.998424 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:06.998314    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="abfa0bd1-0e2e-499c-9c1e-8ede2fc62e4f" containerName="main"
Sep 14 08:09:06.998424 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:06.998330    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="056170e3-8e2a-450f-936d-29fd341e1d79" containerName="istio-proxy"
Sep 14 08:09:07.000368 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.000345    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:07.002333 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.002284    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b69a34\"/\"distinct-w4-b69a34-kserve-self-signed-certs\""
Sep 14 08:09:07.011403 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.011377    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"]
Sep 14 08:09:07.016260 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.016226    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:07.052319 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.052264    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:09:07.118480 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.118440    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-model-cache\") pod \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") "
Sep 14 08:09:07.118684 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.118548    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-tmp-dir\") pod \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") "
Sep 14 08:09:07.118684 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.118566    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-dshm\") pod \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") "
Sep 14 08:09:07.118684 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.118581    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-home\") pod \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") "
Sep 14 08:09:07.118684 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.118628    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gshvl\" (UniqueName: \"kubernetes.io/projected/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-kube-api-access-gshvl\") pod \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") "
Sep 14 08:09:07.118684 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.118655    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-tls-certs\") pod \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\" (UID: \"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0\") "
Sep 14 08:09:07.118957 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.118772    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-model-cache" (OuterVolumeSpecName: "model-cache") pod "6ac18e99-fc19-4a4f-9f66-6cd4893ceae0" (UID: "6ac18e99-fc19-4a4f-9f66-6cd4893ceae0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:07.118957 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.118787    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-dshm\") pod \"distinct-w4-b69a34-kserve-567b4c7f98-lfdhq\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:07.118957 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.118895    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-home" (OuterVolumeSpecName: "home") pod "6ac18e99-fc19-4a4f-9f66-6cd4893ceae0" (UID: "6ac18e99-fc19-4a4f-9f66-6cd4893ceae0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:07.118957 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.118910    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5020f734-0410-4125-857a-f16ba73d8567-tls-certs\") pod \"distinct-w4-b69a34-kserve-567b4c7f98-lfdhq\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:07.118957 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.118905    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6ac18e99-fc19-4a4f-9f66-6cd4893ceae0" (UID: "6ac18e99-fc19-4a4f-9f66-6cd4893ceae0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:07.119213 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.118995    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-home\") pod \"distinct-w4-b69a34-kserve-567b4c7f98-lfdhq\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:07.119213 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.119038    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xmrf\" (UniqueName: \"kubernetes.io/projected/5020f734-0410-4125-857a-f16ba73d8567-kube-api-access-6xmrf\") pod \"distinct-w4-b69a34-kserve-567b4c7f98-lfdhq\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:07.119213 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.119158    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-model-cache\") pod \"distinct-w4-b69a34-kserve-567b4c7f98-lfdhq\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:07.119213 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.119192    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-tmp-dir\") pod \"distinct-w4-b69a34-kserve-567b4c7f98-lfdhq\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:07.119428 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.119243    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-model-cache\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:09:07.119428 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.119254    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-tmp-dir\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:09:07.119428 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.119262    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-home\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:09:07.120991 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.120958    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6ac18e99-fc19-4a4f-9f66-6cd4893ceae0" (UID: "6ac18e99-fc19-4a4f-9f66-6cd4893ceae0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:09:07.121093 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.120983    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-kube-api-access-gshvl" (OuterVolumeSpecName: "kube-api-access-gshvl") pod "6ac18e99-fc19-4a4f-9f66-6cd4893ceae0" (UID: "6ac18e99-fc19-4a4f-9f66-6cd4893ceae0"). InnerVolumeSpecName "kube-api-access-gshvl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:07.121093 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.121000    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-dshm" (OuterVolumeSpecName: "dshm") pod "6ac18e99-fc19-4a4f-9f66-6cd4893ceae0" (UID: "6ac18e99-fc19-4a4f-9f66-6cd4893ceae0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:07.220635 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.220586    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5020f734-0410-4125-857a-f16ba73d8567-tls-certs\") pod \"distinct-w4-b69a34-kserve-567b4c7f98-lfdhq\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:07.220849 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.220661    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-home\") pod \"distinct-w4-b69a34-kserve-567b4c7f98-lfdhq\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:07.220849 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.220703    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xmrf\" (UniqueName: \"kubernetes.io/projected/5020f734-0410-4125-857a-f16ba73d8567-kube-api-access-6xmrf\") pod \"distinct-w4-b69a34-kserve-567b4c7f98-lfdhq\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:07.220849 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.220816    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-model-cache\") pod \"distinct-w4-b69a34-kserve-567b4c7f98-lfdhq\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:07.220849 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.220838    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-tmp-dir\") pod \"distinct-w4-b69a34-kserve-567b4c7f98-lfdhq\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:07.221065 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.220882    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-dshm\") pod \"distinct-w4-b69a34-kserve-567b4c7f98-lfdhq\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:07.221065 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.220947    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-dshm\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:09:07.221065 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.220963    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gshvl\" (UniqueName: \"kubernetes.io/projected/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-kube-api-access-gshvl\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:09:07.221065 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.220979    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0-tls-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:09:07.221065 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.221021    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-home\") pod \"distinct-w4-b69a34-kserve-567b4c7f98-lfdhq\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:07.221219 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.221141    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-model-cache\") pod \"distinct-w4-b69a34-kserve-567b4c7f98-lfdhq\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:07.221251 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.221214    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-tmp-dir\") pod \"distinct-w4-b69a34-kserve-567b4c7f98-lfdhq\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:07.223036 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.223015    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-dshm\") pod \"distinct-w4-b69a34-kserve-567b4c7f98-lfdhq\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:07.223195 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.223174    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5020f734-0410-4125-857a-f16ba73d8567-tls-certs\") pod \"distinct-w4-b69a34-kserve-567b4c7f98-lfdhq\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:07.229889 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.229865    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xmrf\" (UniqueName: \"kubernetes.io/projected/5020f734-0410-4125-857a-f16ba73d8567-kube-api-access-6xmrf\") pod \"distinct-w4-b69a34-kserve-567b4c7f98-lfdhq\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:07.311373 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.311289    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:07.461491 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.461458    2562 generic.go:356] "Generic (PLEG): container finished" podID="6ac18e99-fc19-4a4f-9f66-6cd4893ceae0" containerID="4aac3c4bdaf8728346d689b197dc9b5fe7e096e5b3cd662d7545c86efe69bafa" exitCode=0
Sep 14 08:09:07.461671 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.461510    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"
Sep 14 08:09:07.461671 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.461544    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h" event={"ID":"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0","Type":"ContainerDied","Data":"4aac3c4bdaf8728346d689b197dc9b5fe7e096e5b3cd662d7545c86efe69bafa"}
Sep 14 08:09:07.461671 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.461591    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h" event={"ID":"6ac18e99-fc19-4a4f-9f66-6cd4893ceae0","Type":"ContainerDied","Data":"5e0025bd4b6bf20830cc19f9ea194f7529c61eef22618f9cc5c2f6ccde575a0e"}
Sep 14 08:09:07.461671 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.461612    2562 scope.go:117] "RemoveContainer" containerID="4aac3c4bdaf8728346d689b197dc9b5fe7e096e5b3cd662d7545c86efe69bafa"
Sep 14 08:09:07.477561 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.477483    2562 scope.go:117] "RemoveContainer" containerID="4aac3c4bdaf8728346d689b197dc9b5fe7e096e5b3cd662d7545c86efe69bafa"
Sep 14 08:09:07.477984 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:09:07.477954    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4aac3c4bdaf8728346d689b197dc9b5fe7e096e5b3cd662d7545c86efe69bafa\": container with ID starting with 4aac3c4bdaf8728346d689b197dc9b5fe7e096e5b3cd662d7545c86efe69bafa not found: ID does not exist" containerID="4aac3c4bdaf8728346d689b197dc9b5fe7e096e5b3cd662d7545c86efe69bafa"
Sep 14 08:09:07.478084 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.477990    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4aac3c4bdaf8728346d689b197dc9b5fe7e096e5b3cd662d7545c86efe69bafa"} err="failed to get container status \"4aac3c4bdaf8728346d689b197dc9b5fe7e096e5b3cd662d7545c86efe69bafa\": rpc error: code = NotFound desc = could not find container \"4aac3c4bdaf8728346d689b197dc9b5fe7e096e5b3cd662d7545c86efe69bafa\": container with ID starting with 4aac3c4bdaf8728346d689b197dc9b5fe7e096e5b3cd662d7545c86efe69bafa not found: ID does not exist"
Sep 14 08:09:07.491019 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.490986    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"]
Sep 14 08:09:07.495074 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.495048    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/e2e-conflict-b-cad94a07-kserve-6796d86b9b-xcf8h"]
Sep 14 08:09:07.582031 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:07.582003    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"]
Sep 14 08:09:07.584029 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:09:07.583997    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5020f734_0410_4125_857a_f16ba73d8567.slice/crio-47e407f21817906156b793bec9f0c2678a0ecd50421a1ce16dcc5961b7a79adc WatchSource:0}: Error finding container 47e407f21817906156b793bec9f0c2678a0ecd50421a1ce16dcc5961b7a79adc: Status 404 returned error can't find the container with id 47e407f21817906156b793bec9f0c2678a0ecd50421a1ce16dcc5961b7a79adc
Sep 14 08:09:08.113548 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:08.113513    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ac18e99-fc19-4a4f-9f66-6cd4893ceae0" path="/var/lib/kubelet/pods/6ac18e99-fc19-4a4f-9f66-6cd4893ceae0/volumes"
Sep 14 08:09:08.468986 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:08.468948    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq" event={"ID":"5020f734-0410-4125-857a-f16ba73d8567","Type":"ContainerStarted","Data":"be067c52971278fbefa75ca71f889033b41b9d9842f8dd26e0fbdb86768eb46d"}
Sep 14 08:09:08.469176 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:08.469000    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq" event={"ID":"5020f734-0410-4125-857a-f16ba73d8567","Type":"ContainerStarted","Data":"47e407f21817906156b793bec9f0c2678a0ecd50421a1ce16dcc5961b7a79adc"}
Sep 14 08:09:08.487832 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:08.487773    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq" podStartSLOduration=2.487757003 podStartE2EDuration="2.487757003s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:09:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:09:08.485989396 +0000 UTC m=+1250.942439395" watchObservedRunningTime="2026-09-14 08:09:08.487757003 +0000 UTC m=+1250.944207004"
Sep 14 08:09:11.405490 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:11.405410    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:17.312057 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:17.312019    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:17.312556 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:17.312074    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:17.324986 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:17.324958    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:17.530021 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:17.529978    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:09:18.103414 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:18.103381    2562 scope.go:117] "RemoveContainer" containerID="24b8d99be711ebb8c6d2f2b5d3d9d50a03e690326fc3ad9895cfed949844bdd6"
Sep 14 08:09:18.111888 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:18.111864    2562 scope.go:117] "RemoveContainer" containerID="e026fe8ccc6880541df040e4916349665d21f45cd2e3f2be4b6c99cf5538a7e5"
Sep 14 08:09:19.571585 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:19.571543    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8b87debd-dt49k"]
Sep 14 08:09:19.572489 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:19.572460    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ac18e99-fc19-4a4f-9f66-6cd4893ceae0" containerName="main"
Sep 14 08:09:19.572489 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:19.572489    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ac18e99-fc19-4a4f-9f66-6cd4893ceae0" containerName="main"
Sep 14 08:09:19.572752 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:19.572738    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ac18e99-fc19-4a4f-9f66-6cd4893ceae0" containerName="main"
Sep 14 08:09:19.578445 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:19.578415    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8b87debd-dt49k"
Sep 14 08:09:19.580313 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:19.580284    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8b87debd-dt49k"]
Sep 14 08:09:19.580856 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:19.580830    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:09:19.580972 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:19.580879    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:09:19.750548 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:19.750475    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6687f986-310d-45c0-a52b-744bf7a910a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8b87debd-dt49k\" (UID: \"6687f986-310d-45c0-a52b-744bf7a910a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8b87debd-dt49k"
Sep 14 08:09:19.852034 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:19.851943    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6687f986-310d-45c0-a52b-744bf7a910a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8b87debd-dt49k\" (UID: \"6687f986-310d-45c0-a52b-744bf7a910a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8b87debd-dt49k"
Sep 14 08:09:19.852586 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:19.852568    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6687f986-310d-45c0-a52b-744bf7a910a8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-8b87debd-dt49k\" (UID: \"6687f986-310d-45c0-a52b-744bf7a910a8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8b87debd-dt49k"
Sep 14 08:09:19.890900 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:19.890866    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8b87debd-dt49k"
Sep 14 08:09:19.904662 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:19.904625    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:20.161233 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:20.161203    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8b87debd-dt49k"]
Sep 14 08:09:20.163267 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:09:20.163237    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6687f986_310d_45c0_a52b_744bf7a910a8.slice/crio-66d003170fd4d1a2b6b62875a4228846a9f7949680732effd96d073aad6c3347 WatchSource:0}: Error finding container 66d003170fd4d1a2b6b62875a4228846a9f7949680732effd96d073aad6c3347: Status 404 returned error can't find the container with id 66d003170fd4d1a2b6b62875a4228846a9f7949680732effd96d073aad6c3347
Sep 14 08:09:20.535582 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:20.535543    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8b87debd-dt49k" event={"ID":"6687f986-310d-45c0-a52b-744bf7a910a8","Type":"ContainerStarted","Data":"65747a3d7aaaa22115b7bba3c37d7d9492d00274cadc52348cf4554ee43dbd48"}
Sep 14 08:09:20.535845 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:20.535612    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8b87debd-dt49k" event={"ID":"6687f986-310d-45c0-a52b-744bf7a910a8","Type":"ContainerStarted","Data":"66d003170fd4d1a2b6b62875a4228846a9f7949680732effd96d073aad6c3347"}
Sep 14 08:09:20.553036 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:20.552974    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8b87debd-dt49k" podStartSLOduration=1.552957652 podStartE2EDuration="1.552957652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:09:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:09:20.551055496 +0000 UTC m=+1263.007505490" watchObservedRunningTime="2026-09-14 08:09:20.552957652 +0000 UTC m=+1263.009407645"
Sep 14 08:09:20.646849 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:20.646812    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:21.499911 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:21.499877    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:21.542491 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:21.542452    2562 generic.go:356] "Generic (PLEG): container finished" podID="6687f986-310d-45c0-a52b-744bf7a910a8" containerID="65747a3d7aaaa22115b7bba3c37d7d9492d00274cadc52348cf4554ee43dbd48" exitCode=0
Sep 14 08:09:21.542679 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:21.542544    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8b87debd-dt49k" event={"ID":"6687f986-310d-45c0-a52b-744bf7a910a8","Type":"ContainerDied","Data":"65747a3d7aaaa22115b7bba3c37d7d9492d00274cadc52348cf4554ee43dbd48"}
Sep 14 08:09:21.633844 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:21.633811    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:22.250117 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:22.250084    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:22.675560 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:22.675535    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8b87debd-dt49k"
Sep 14 08:09:22.785261 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:22.785223    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6687f986-310d-45c0-a52b-744bf7a910a8-maas-api-service-ca\") pod \"6687f986-310d-45c0-a52b-744bf7a910a8\" (UID: \"6687f986-310d-45c0-a52b-744bf7a910a8\") "
Sep 14 08:09:22.785645 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:22.785621    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6687f986-310d-45c0-a52b-744bf7a910a8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6687f986-310d-45c0-a52b-744bf7a910a8" (UID: "6687f986-310d-45c0-a52b-744bf7a910a8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:09:22.886750 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:22.886666    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6687f986-310d-45c0-a52b-744bf7a910a8-maas-api-service-ca\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:09:22.976180 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:22.976148    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:23.553740 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:23.553705    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8b87debd-dt49k"
Sep 14 08:09:23.553740 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:23.553714    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8b87debd-dt49k" event={"ID":"6687f986-310d-45c0-a52b-744bf7a910a8","Type":"ContainerDied","Data":"66d003170fd4d1a2b6b62875a4228846a9f7949680732effd96d073aad6c3347"}
Sep 14 08:09:23.554196 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:23.553768    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="66d003170fd4d1a2b6b62875a4228846a9f7949680732effd96d073aad6c3347"
Sep 14 08:09:24.184924 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:24.184888    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:24.786271 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:24.786234    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:25.525253 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:25.525217    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:09:26.036827 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:26.036793    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:35.287298 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:35.287259    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:35.378726 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:35.378691    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:35.437425 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:35.437388    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:35.525052 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:35.525002    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8b87debd-dt49k"]
Sep 14 08:09:35.528154 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:35.528112    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:35.528489 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:35.528471    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-8b87debd-dt49k"]
Sep 14 08:09:35.586012 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:35.585894    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:36.014472 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:36.014433    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:36.114169 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:36.114130    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6687f986-310d-45c0-a52b-744bf7a910a8" path="/var/lib/kubelet/pods/6687f986-310d-45c0-a52b-744bf7a910a8/volumes"
Sep 14 08:09:36.745643 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:36.745594    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:37.432561 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:37.432482    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:38.251255 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:38.251220    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:38.600961 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:38.600878    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:39.033076 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:39.033037    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:39.471190 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:39.471158    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:40.031321 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:40.031272    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:43.580410 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:43.580367    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"]
Sep 14 08:09:43.622881 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:43.622849    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:46.796635 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:46.796601    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"]
Sep 14 08:09:46.797029 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:46.796986    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6687f986-310d-45c0-a52b-744bf7a910a8" containerName="revoke-keys"
Sep 14 08:09:46.797029 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:46.796997    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6687f986-310d-45c0-a52b-744bf7a910a8" containerName="revoke-keys"
Sep 14 08:09:46.797112 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:46.797080    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6687f986-310d-45c0-a52b-744bf7a910a8" containerName="revoke-keys"
Sep 14 08:09:46.798892 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:46.798867    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:46.801606 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:46.801578    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b69a34\"/\"distinct-2-w4-b69a34-kserve-self-signed-certs\""
Sep 14 08:09:46.881976 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:46.881933    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:46.882149 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:46.882025    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"]
Sep 14 08:09:46.934088 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:46.934044    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-model-cache\") pod \"distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:46.934284 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:46.934158    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmzn7\" (UniqueName: \"kubernetes.io/projected/4a933732-a923-45d2-afc3-e802f0c8cbfa-kube-api-access-gmzn7\") pod \"distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:46.934284 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:46.934187    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-tmp-dir\") pod \"distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:46.934375 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:46.934273    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a933732-a923-45d2-afc3-e802f0c8cbfa-tls-certs\") pod \"distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:46.934375 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:46.934304    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-home\") pod \"distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:46.934375 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:46.934328    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-dshm\") pod \"distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:47.035323 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:47.035279    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmzn7\" (UniqueName: \"kubernetes.io/projected/4a933732-a923-45d2-afc3-e802f0c8cbfa-kube-api-access-gmzn7\") pod \"distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:47.035566 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:47.035338    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-tmp-dir\") pod \"distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:47.035566 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:47.035388    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a933732-a923-45d2-afc3-e802f0c8cbfa-tls-certs\") pod \"distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:47.035566 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:47.035415    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-home\") pod \"distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:47.035762 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:47.035568    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-dshm\") pod \"distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:47.035762 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:47.035623    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-model-cache\") pod \"distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:47.035868 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:47.035846    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-tmp-dir\") pod \"distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:47.035927 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:47.035866    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-home\") pod \"distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:47.035975 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:47.035932    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-model-cache\") pod \"distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:47.037776 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:47.037753    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-dshm\") pod \"distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:47.038204 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:47.038184    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a933732-a923-45d2-afc3-e802f0c8cbfa-tls-certs\") pod \"distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:47.043930 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:47.043906    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmzn7\" (UniqueName: \"kubernetes.io/projected/4a933732-a923-45d2-afc3-e802f0c8cbfa-kube-api-access-gmzn7\") pod \"distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") " pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:47.108268 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:47.108164    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:47.383287 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:47.383255    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"]
Sep 14 08:09:47.385994 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:09:47.385961    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a933732_a923_45d2_afc3_e802f0c8cbfa.slice/crio-0e97d1e704f815fd2f8e09c5c7d7883ec2b0fdcc8ec3de5d33242c7135584ee9 WatchSource:0}: Error finding container 0e97d1e704f815fd2f8e09c5c7d7883ec2b0fdcc8ec3de5d33242c7135584ee9: Status 404 returned error can't find the container with id 0e97d1e704f815fd2f8e09c5c7d7883ec2b0fdcc8ec3de5d33242c7135584ee9
Sep 14 08:09:47.678795 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:47.678697    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc" event={"ID":"4a933732-a923-45d2-afc3-e802f0c8cbfa","Type":"ContainerStarted","Data":"871db68cc9d06b0a1e067d3b5cd77ee75df5556b76287c5425e24aeaf97ce1d9"}
Sep 14 08:09:47.678795 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:47.678741    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc" event={"ID":"4a933732-a923-45d2-afc3-e802f0c8cbfa","Type":"ContainerStarted","Data":"0e97d1e704f815fd2f8e09c5c7d7883ec2b0fdcc8ec3de5d33242c7135584ee9"}
Sep 14 08:09:47.696140 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:47.696085    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc" podStartSLOduration=1.696067488 podStartE2EDuration="1.696067488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:09:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:09:47.693932065 +0000 UTC m=+1290.150382058" watchObservedRunningTime="2026-09-14 08:09:47.696067488 +0000 UTC m=+1290.152517509"
Sep 14 08:09:50.306209 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:50.306173    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:09:54.732201 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:54.732161    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:09:57.108565 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:57.108526    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:57.108565 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:57.108574    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:57.121223 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:57.121195    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:57.745292 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:57.745261    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:09:58.600382 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:58.600317    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b" podUID="6d0a01fd-3657-4149-a176-9ee4937cb166" containerName="main" containerID="cri-o://e98b5790fbd20cdf0ebb08ffc799aba630e9bb718e4777e06dc1dc83acc7a7ee" gracePeriod=45
Sep 14 08:09:58.740536 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:58.740476    2562 generic.go:356] "Generic (PLEG): container finished" podID="6d0a01fd-3657-4149-a176-9ee4937cb166" containerID="e98b5790fbd20cdf0ebb08ffc799aba630e9bb718e4777e06dc1dc83acc7a7ee" exitCode=0
Sep 14 08:09:58.740700 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:58.740559    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b" event={"ID":"6d0a01fd-3657-4149-a176-9ee4937cb166","Type":"ContainerDied","Data":"e98b5790fbd20cdf0ebb08ffc799aba630e9bb718e4777e06dc1dc83acc7a7ee"}
Sep 14 08:09:58.847190 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:58.847164    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:09:58.857990 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:58.857937    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2ngb\" (UniqueName: \"kubernetes.io/projected/6d0a01fd-3657-4149-a176-9ee4937cb166-kube-api-access-k2ngb\") pod \"6d0a01fd-3657-4149-a176-9ee4937cb166\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") "
Sep 14 08:09:58.857990 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:58.857980    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-model-cache\") pod \"6d0a01fd-3657-4149-a176-9ee4937cb166\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") "
Sep 14 08:09:58.858124 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:58.858012    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-home\") pod \"6d0a01fd-3657-4149-a176-9ee4937cb166\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") "
Sep 14 08:09:58.858124 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:58.858058    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d0a01fd-3657-4149-a176-9ee4937cb166-tls-certs\") pod \"6d0a01fd-3657-4149-a176-9ee4937cb166\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") "
Sep 14 08:09:58.858124 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:58.858079    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-dshm\") pod \"6d0a01fd-3657-4149-a176-9ee4937cb166\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") "
Sep 14 08:09:58.858267 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:58.858214    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-tmp-dir\") pod \"6d0a01fd-3657-4149-a176-9ee4937cb166\" (UID: \"6d0a01fd-3657-4149-a176-9ee4937cb166\") "
Sep 14 08:09:58.858324 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:58.858303    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-home" (OuterVolumeSpecName: "home") pod "6d0a01fd-3657-4149-a176-9ee4937cb166" (UID: "6d0a01fd-3657-4149-a176-9ee4937cb166"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:58.858385 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:58.858311    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-model-cache" (OuterVolumeSpecName: "model-cache") pod "6d0a01fd-3657-4149-a176-9ee4937cb166" (UID: "6d0a01fd-3657-4149-a176-9ee4937cb166"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:58.858450 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:58.858420    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6d0a01fd-3657-4149-a176-9ee4937cb166" (UID: "6d0a01fd-3657-4149-a176-9ee4937cb166"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:58.858638 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:58.858621    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-home\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:09:58.858719 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:58.858642    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-tmp-dir\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:09:58.858719 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:58.858655    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-model-cache\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:09:58.860161 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:58.860137    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d0a01fd-3657-4149-a176-9ee4937cb166-kube-api-access-k2ngb" (OuterVolumeSpecName: "kube-api-access-k2ngb") pod "6d0a01fd-3657-4149-a176-9ee4937cb166" (UID: "6d0a01fd-3657-4149-a176-9ee4937cb166"). InnerVolumeSpecName "kube-api-access-k2ngb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:58.860567 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:58.860544    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-dshm" (OuterVolumeSpecName: "dshm") pod "6d0a01fd-3657-4149-a176-9ee4937cb166" (UID: "6d0a01fd-3657-4149-a176-9ee4937cb166"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:58.860567 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:58.860552    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d0a01fd-3657-4149-a176-9ee4937cb166-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6d0a01fd-3657-4149-a176-9ee4937cb166" (UID: "6d0a01fd-3657-4149-a176-9ee4937cb166"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:09:58.959335 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:58.959295    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d0a01fd-3657-4149-a176-9ee4937cb166-tls-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:09:58.959335 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:58.959330    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d0a01fd-3657-4149-a176-9ee4937cb166-dshm\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:09:58.959335 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:58.959340    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2ngb\" (UniqueName: \"kubernetes.io/projected/6d0a01fd-3657-4149-a176-9ee4937cb166-kube-api-access-k2ngb\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:09:59.747398 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:59.747366    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"
Sep 14 08:09:59.747897 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:59.747409    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b" event={"ID":"6d0a01fd-3657-4149-a176-9ee4937cb166","Type":"ContainerDied","Data":"6493eb4e32ea7ed3e797932cb34406acfdf546d6e1d748efd058e010fffc4df0"}
Sep 14 08:09:59.747897 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:59.747471    2562 scope.go:117] "RemoveContainer" containerID="e98b5790fbd20cdf0ebb08ffc799aba630e9bb718e4777e06dc1dc83acc7a7ee"
Sep 14 08:09:59.768837 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:59.768808    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"]
Sep 14 08:09:59.773136 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:09:59.773108    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-8b87debd/e2e-lifecycle-model-8b87debd-kserve-77964d6979-m4g4b"]
Sep 14 08:10:00.112704 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:00.112616    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d0a01fd-3657-4149-a176-9ee4937cb166" path="/var/lib/kubelet/pods/6d0a01fd-3657-4149-a176-9ee4937cb166/volumes"
Sep 14 08:10:15.601228 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:15.601191    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:10:18.146641 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:18.146609    2562 scope.go:117] "RemoveContainer" containerID="073889c23e1015c7fe2a1235341734c96769ad55dd4a03acbfa11a15b83d4e36"
Sep 14 08:10:20.890871 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:20.890833    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:21.104442 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:21.104407    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:26.606312 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:26.606276    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:26.660369 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:26.660331    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:26.858040 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:26.857952    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:28.122251 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:28.122213    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:28.230550 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:28.230512    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:28.330970 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:28.330933    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:28.902384 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:28.902351    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:29.002133 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:29.002089    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:29.103883 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:29.103846    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:30.381552 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:30.381524    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:34.099407 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:34.099362    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:10:36.215576 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:36.215534    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:36.319547 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:36.319496    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:36.850634 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:36.850578    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:42.863601 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:42.863462    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:42.967019 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:42.966976    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:43.064415 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:43.064356    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:53.522837 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:53.522799    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:53.749326 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:53.749289    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:10:58.617809 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:10:58.617774    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:11:14.678300 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:14.678260    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:20.538183 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:20.538147    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:20.591090 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:20.591060    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:20.880329 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:20.880215    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:21.170046 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.169962    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"]
Sep 14 08:11:21.170577 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.170559    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d0a01fd-3657-4149-a176-9ee4937cb166" containerName="main"
Sep 14 08:11:21.170577 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.170578    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d0a01fd-3657-4149-a176-9ee4937cb166" containerName="main"
Sep 14 08:11:21.170716 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.170704    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d0a01fd-3657-4149-a176-9ee4937cb166" containerName="main"
Sep 14 08:11:21.174087 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.174066    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.176095 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.176071    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-b914f7-openshift-default-dockercfg-tk7bm\""
Sep 14 08:11:21.182630 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.182266    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"]
Sep 14 08:11:21.196916 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.196882    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:21.316238 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.316198    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-workload-socket\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.316238 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.316235    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-credential-socket\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.316545 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.316263    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-podinfo\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.316545 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.316392    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-data\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.316686 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.316551    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-token\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.316748 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.316686    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istiod-ca-cert\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.316807 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.316775    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m24zq\" (UniqueName: \"kubernetes.io/projected/39b6749e-8f5c-460e-bea4-1c0c44b053ae-kube-api-access-m24zq\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.316896 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.316838    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-envoy\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.316959 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.316915    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-workload-certs\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.351110 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.351078    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:21.417991 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.417947    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m24zq\" (UniqueName: \"kubernetes.io/projected/39b6749e-8f5c-460e-bea4-1c0c44b053ae-kube-api-access-m24zq\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.418638 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.418611    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-envoy\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.419183 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.419163    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-workload-certs\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.419717 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.419697    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-workload-socket\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.420229 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.420176    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-credential-socket\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.420710 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.420690    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-podinfo\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.421332 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.420613    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-credential-socket\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.421332 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.420112    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-workload-socket\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.421332 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.419619    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-workload-certs\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.421666 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.421646    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-data\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.421967 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.421899    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-token\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.422217 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.422197    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istiod-ca-cert\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.423067 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.423000    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istiod-ca-cert\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.423193 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.422035    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-data\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.423311 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.423247    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-envoy\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.424059 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.424030    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-podinfo\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.429272 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.429246    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-token\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.433782 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.433751    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m24zq\" (UniqueName: \"kubernetes.io/projected/39b6749e-8f5c-460e-bea4-1c0c44b053ae-kube-api-access-m24zq\") pod \"e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") " pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.487475 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.487438    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:21.766108 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.766080    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"]
Sep 14 08:11:21.768411 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:11:21.768382    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39b6749e_8f5c_460e_bea4_1c0c44b053ae.slice/crio-11d7ce2fd9f0dee06980e038a5adf4a198940450b596bda4606de2d35c0b96a3 WatchSource:0}: Error finding container 11d7ce2fd9f0dee06980e038a5adf4a198940450b596bda4606de2d35c0b96a3: Status 404 returned error can't find the container with id 11d7ce2fd9f0dee06980e038a5adf4a198940450b596bda4606de2d35c0b96a3
Sep 14 08:11:21.770627 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.770594    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:11:21.770749 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.770665    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:11:21.770749 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:21.770693    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:11:22.180098 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:22.180005    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4" event={"ID":"39b6749e-8f5c-460e-bea4-1c0c44b053ae","Type":"ContainerStarted","Data":"6e867094c2ec75ae092c24e567e2c07c7533c4f5920241b383a86d7fa4e80e55"}
Sep 14 08:11:22.180098 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:22.180089    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4" event={"ID":"39b6749e-8f5c-460e-bea4-1c0c44b053ae","Type":"ContainerStarted","Data":"11d7ce2fd9f0dee06980e038a5adf4a198940450b596bda4606de2d35c0b96a3"}
Sep 14 08:11:22.200881 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:22.200822    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4" podStartSLOduration=1.200807261 podStartE2EDuration="1.200807261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:11:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:11:22.199274419 +0000 UTC m=+1384.655724424" watchObservedRunningTime="2026-09-14 08:11:22.200807261 +0000 UTC m=+1384.657257253"
Sep 14 08:11:22.409711 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:22.409676    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:11:22.488235 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:22.488198    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:22.492959 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:22.492932    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:23.189233 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:23.189203    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:23.191063 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:23.191036    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:23.294375 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:23.294339    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:23.524669 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:23.524486    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:24.242547 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:24.242517    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:24.342175 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:24.342139    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:24.543972 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:24.543871    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:25.262425 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:25.262388    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:26.221089 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:26.221055    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:26.921686 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:26.921641    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:27.612967 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:27.612931    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:28.374953 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:28.374908    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:28.478894 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:28.478853    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:28.669433 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:28.669346    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:28.977066 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:28.977027    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:29.066891 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:29.066861    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:34.484167 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:34.484125    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:41.333443 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:41.333413    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:47.316183 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:47.316101    2562 generic.go:356] "Generic (PLEG): container finished" podID="39b6749e-8f5c-460e-bea4-1c0c44b053ae" containerID="6e867094c2ec75ae092c24e567e2c07c7533c4f5920241b383a86d7fa4e80e55" exitCode=0
Sep 14 08:11:47.316629 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:47.316176    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4" event={"ID":"39b6749e-8f5c-460e-bea4-1c0c44b053ae","Type":"ContainerDied","Data":"6e867094c2ec75ae092c24e567e2c07c7533c4f5920241b383a86d7fa4e80e55"}
Sep 14 08:11:47.316695 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:47.316678    2562 scope.go:117] "RemoveContainer" containerID="6e867094c2ec75ae092c24e567e2c07c7533c4f5920241b383a86d7fa4e80e55"
Sep 14 08:11:47.317946 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:47.317909    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:11:47.318063 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:47.317984    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:11:47.318063 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:47.318022    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 14 08:11:48.323481 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:48.323445    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4" event={"ID":"39b6749e-8f5c-460e-bea4-1c0c44b053ae","Type":"ContainerStarted","Data":"d6f25cbda052875dcee896f642bf6fa5edb3c8046c0189c3e8ed8ea2467f28f8"}
Sep 14 08:11:48.488410 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:48.488375    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:48.489513 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:48.489480    2562 patch_prober.go:28] interesting pod/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 14 08:11:48.489645 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:48.489568    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4" podUID="39b6749e-8f5c-460e-bea4-1c0c44b053ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 14 08:11:49.344384 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:49.344350    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:49.488222 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:49.488183    2562 patch_prober.go:28] interesting pod/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 14 08:11:49.488409 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:49.488244    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4" podUID="39b6749e-8f5c-460e-bea4-1c0c44b053ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 14 08:11:50.488384 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:50.488344    2562 patch_prober.go:28] interesting pod/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 14 08:11:50.488848 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:50.488405    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4" podUID="39b6749e-8f5c-460e-bea4-1c0c44b053ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 14 08:11:50.819874 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:50.819791    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:51.487954 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:51.487910    2562 patch_prober.go:28] interesting pod/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 14 08:11:51.488170 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:51.487988    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4" podUID="39b6749e-8f5c-460e-bea4-1c0c44b053ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 14 08:11:51.559600 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:51.559554    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:51.607990 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:51.607944    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:11:52.293426 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:52.293393    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:52.488373 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:52.488331    2562 patch_prober.go:28] interesting pod/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 14 08:11:52.488584 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:52.488399    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4" podUID="39b6749e-8f5c-460e-bea4-1c0c44b053ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 14 08:11:53.144459 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:53.144409    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:53.190038 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:53.189988    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:53.519159 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:53.519118    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:53.520734 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:53.520709    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:11:53.971548 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:53.971489    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:54.238117 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:54.238031    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:54.712408 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:54.712360    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:54.812775 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:54.812741    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:55.023909 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:55.023815    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:55.412615 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:55.412521    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:55.605336 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:55.605306    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:57.213366 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:57.213323    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:58.027322 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:58.027282    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:58.118543 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:58.118489    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:58.217770 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:58.217740    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:59.134863 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:59.134829    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:11:59.901083 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:11:59.901051    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:00.600275 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:00.600223    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:01.344286 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:01.344240    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:02.068273 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:02.068234    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:02.914287 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:02.914246    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"]
Sep 14 08:12:02.917380 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:02.917357    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"
Sep 14 08:12:02.920247 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:02.919994    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-87b7cb-serving-cert\""
Sep 14 08:12:02.920247 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:02.920077    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:12:02.920247 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:02.920124    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-g2t2q\""
Sep 14 08:12:02.920529 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:02.920382    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 08:12:02.920593 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:02.920562    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 08:12:02.928869 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:02.928845    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"]
Sep 14 08:12:02.943028 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:02.942991    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:03.044354 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:03.044320    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:03.050428 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:03.050402    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-openshift-service-ca\") pod \"maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2\" (UID: \"b8fc99a0-16f5-4cfb-ac28-569ec936bcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"
Sep 14 08:12:03.050573 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:03.050444    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxbq6\" (UniqueName: \"kubernetes.io/projected/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-kube-api-access-gxbq6\") pod \"maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2\" (UID: \"b8fc99a0-16f5-4cfb-ac28-569ec936bcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"
Sep 14 08:12:03.050641 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:03.050622    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-maas-api-tls\") pod \"maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2\" (UID: \"b8fc99a0-16f5-4cfb-ac28-569ec936bcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"
Sep 14 08:12:03.050699 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:03.050682    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-metrics-certs\") pod \"maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2\" (UID: \"b8fc99a0-16f5-4cfb-ac28-569ec936bcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"
Sep 14 08:12:03.144453 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:03.144412    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:03.152057 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:03.152038    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-maas-api-tls\") pod \"maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2\" (UID: \"b8fc99a0-16f5-4cfb-ac28-569ec936bcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"
Sep 14 08:12:03.152117 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:03.152078    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-metrics-certs\") pod \"maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2\" (UID: \"b8fc99a0-16f5-4cfb-ac28-569ec936bcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"
Sep 14 08:12:03.152170 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:03.152122    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-openshift-service-ca\") pod \"maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2\" (UID: \"b8fc99a0-16f5-4cfb-ac28-569ec936bcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"
Sep 14 08:12:03.152170 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:03.152143    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxbq6\" (UniqueName: \"kubernetes.io/projected/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-kube-api-access-gxbq6\") pod \"maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2\" (UID: \"b8fc99a0-16f5-4cfb-ac28-569ec936bcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"
Sep 14 08:12:03.152856 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:03.152834    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-openshift-service-ca\") pod \"maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2\" (UID: \"b8fc99a0-16f5-4cfb-ac28-569ec936bcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"
Sep 14 08:12:03.154755 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:03.154732    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-maas-api-tls\") pod \"maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2\" (UID: \"b8fc99a0-16f5-4cfb-ac28-569ec936bcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"
Sep 14 08:12:03.154834 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:03.154732    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-metrics-certs\") pod \"maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2\" (UID: \"b8fc99a0-16f5-4cfb-ac28-569ec936bcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"
Sep 14 08:12:03.163837 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:03.163810    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxbq6\" (UniqueName: \"kubernetes.io/projected/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-kube-api-access-gxbq6\") pod \"maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2\" (UID: \"b8fc99a0-16f5-4cfb-ac28-569ec936bcce\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"
Sep 14 08:12:03.232872 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:03.232832    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"
Sep 14 08:12:03.438075 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:03.438039    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:03.505740 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:03.505709    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"]
Sep 14 08:12:03.506763 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:03.506740    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:03.507417 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:12:03.507393    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8fc99a0_16f5_4cfb_ac28_569ec936bcce.slice/crio-61ef5e86999c2b64d254a68315d673c43397375dc265a7541f3097a0798f84bb WatchSource:0}: Error finding container 61ef5e86999c2b64d254a68315d673c43397375dc265a7541f3097a0798f84bb: Status 404 returned error can't find the container with id 61ef5e86999c2b64d254a68315d673c43397375dc265a7541f3097a0798f84bb
Sep 14 08:12:03.508868 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:03.508851    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 08:12:04.417256 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:04.417187    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2" event={"ID":"b8fc99a0-16f5-4cfb-ac28-569ec936bcce","Type":"ContainerStarted","Data":"61ef5e86999c2b64d254a68315d673c43397375dc265a7541f3097a0798f84bb"}
Sep 14 08:12:06.431153 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:06.431112    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2" event={"ID":"b8fc99a0-16f5-4cfb-ac28-569ec936bcce","Type":"ContainerStarted","Data":"d6fdb5920d5794b26f6335a410aac82f76381d62c9358ea6fd2341343c8a7d8c"}
Sep 14 08:12:06.431699 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:06.431182    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"
Sep 14 08:12:06.451737 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:06.451686    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2" podStartSLOduration=2.154550808 podStartE2EDuration="4.451669771s" totalImagesPullingTime="2.297118963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:12:06.449371741 +0000 UTC m=+1428.905821746" watchObservedRunningTime="2026-09-14 08:12:06.451669771 +0000 UTC m=+1428.908119758"
Sep 14 08:12:09.101619 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:09.101580    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:09.603945 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:09.603904    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:10.553951 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:10.553870    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:11.443366 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:11.443329    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:12.442967 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:12.442939    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"
Sep 14 08:12:12.525992 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:12.525943    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:13.267867 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:13.267830    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:14.080759 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:14.080726    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"]
Sep 14 08:12:14.085744 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:14.085717    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"]
Sep 14 08:12:14.112942 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:14.112915    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"]
Sep 14 08:12:14.175200 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:14.175162    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:14.324080 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:14.324044    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:15.031464 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:15.031424    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:15.085377 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:15.085345    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:15.383676 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:15.383581    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:16.119066 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:16.119034    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:16.919207 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:16.919168    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:17.770378 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:17.770340    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:17.970191 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:17.970152    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:18.179211 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:18.179131    2562 scope.go:117] "RemoveContainer" containerID="d8023db50ee82272a1d23abacf735ea6570b9d1ceb1fbaeaa247db229fb3d072"
Sep 14 08:12:18.989626 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:18.989584    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:19.724162 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:19.724119    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:12:19.984996 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:19.984912    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:21.230249 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:21.230214    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:22.057369 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:22.057339    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:23.125954 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:23.125922    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:23.951312 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:23.951275    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:24.812123 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:24.812082    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:25.538470 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:25.538433    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:26.234732 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:26.234702    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:27.132049 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:27.132012    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:27.851389 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:27.851356    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:27.935870 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:27.935833    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:27.989318 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:27.989280    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:28.066610 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:28.066574    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:28.146874 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:28.146779    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:28.434061 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:28.434028    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:28.553000 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:28.552964    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:29.100401 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.100361    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc" podUID="4a933732-a923-45d2-afc3-e802f0c8cbfa" containerName="main" containerID="cri-o://871db68cc9d06b0a1e067d3b5cd77ee75df5556b76287c5425e24aeaf97ce1d9" gracePeriod=45
Sep 14 08:12:29.104814 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.104780    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq" podUID="5020f734-0410-4125-857a-f16ba73d8567" containerName="main" containerID="cri-o://be067c52971278fbefa75ca71f889033b41b9d9842f8dd26e0fbdb86768eb46d" gracePeriod=45
Sep 14 08:12:29.133303 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.133266    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt" podUID="bd70e9fd-10f3-4b03-be62-7e130ad88022" containerName="main" containerID="cri-o://29c18e1a93f9b8ff81ca6c24fd5f9414a8d602ba4d37a525181b13787f2f7db8" gracePeriod=45
Sep 14 08:12:29.289874 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.289830    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:29.401309 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.401277    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:12:29.436375 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.436345    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:12:29.451562 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.451533    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:12:29.520669 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.520625    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xmrf\" (UniqueName: \"kubernetes.io/projected/5020f734-0410-4125-857a-f16ba73d8567-kube-api-access-6xmrf\") pod \"5020f734-0410-4125-857a-f16ba73d8567\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") "
Sep 14 08:12:29.520841 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.520688    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-model-cache\") pod \"bd70e9fd-10f3-4b03-be62-7e130ad88022\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") "
Sep 14 08:12:29.520841 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.520717    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-home\") pod \"5020f734-0410-4125-857a-f16ba73d8567\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") "
Sep 14 08:12:29.520841 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.520758    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d98sl\" (UniqueName: \"kubernetes.io/projected/bd70e9fd-10f3-4b03-be62-7e130ad88022-kube-api-access-d98sl\") pod \"bd70e9fd-10f3-4b03-be62-7e130ad88022\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") "
Sep 14 08:12:29.520841 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.520787    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-tmp-dir\") pod \"5020f734-0410-4125-857a-f16ba73d8567\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") "
Sep 14 08:12:29.520841 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.520821    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-model-cache\") pod \"4a933732-a923-45d2-afc3-e802f0c8cbfa\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") "
Sep 14 08:12:29.521114 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.520874    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-home\") pod \"4a933732-a923-45d2-afc3-e802f0c8cbfa\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") "
Sep 14 08:12:29.521114 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.520938    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-tmp-dir\") pod \"4a933732-a923-45d2-afc3-e802f0c8cbfa\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") "
Sep 14 08:12:29.521114 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.520962    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-dshm\") pod \"5020f734-0410-4125-857a-f16ba73d8567\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") "
Sep 14 08:12:29.521114 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.520990    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5020f734-0410-4125-857a-f16ba73d8567-tls-certs\") pod \"5020f734-0410-4125-857a-f16ba73d8567\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") "
Sep 14 08:12:29.521114 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.521012    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmzn7\" (UniqueName: \"kubernetes.io/projected/4a933732-a923-45d2-afc3-e802f0c8cbfa-kube-api-access-gmzn7\") pod \"4a933732-a923-45d2-afc3-e802f0c8cbfa\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") "
Sep 14 08:12:29.521114 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.521016    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-home" (OuterVolumeSpecName: "home") pod "5020f734-0410-4125-857a-f16ba73d8567" (UID: "5020f734-0410-4125-857a-f16ba73d8567"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:29.521114 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.521032    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-model-cache" (OuterVolumeSpecName: "model-cache") pod "bd70e9fd-10f3-4b03-be62-7e130ad88022" (UID: "bd70e9fd-10f3-4b03-be62-7e130ad88022"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:29.521114 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.521045    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd70e9fd-10f3-4b03-be62-7e130ad88022-tls-certs\") pod \"bd70e9fd-10f3-4b03-be62-7e130ad88022\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") "
Sep 14 08:12:29.521114 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.521092    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-home\") pod \"bd70e9fd-10f3-4b03-be62-7e130ad88022\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") "
Sep 14 08:12:29.521609 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.521134    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-dshm\") pod \"4a933732-a923-45d2-afc3-e802f0c8cbfa\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") "
Sep 14 08:12:29.521609 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.521140    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5020f734-0410-4125-857a-f16ba73d8567" (UID: "5020f734-0410-4125-857a-f16ba73d8567"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:29.521609 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.521164    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-home" (OuterVolumeSpecName: "home") pod "4a933732-a923-45d2-afc3-e802f0c8cbfa" (UID: "4a933732-a923-45d2-afc3-e802f0c8cbfa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:29.521609 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.521197    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-dshm\") pod \"bd70e9fd-10f3-4b03-be62-7e130ad88022\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") "
Sep 14 08:12:29.521609 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.521243    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a933732-a923-45d2-afc3-e802f0c8cbfa-tls-certs\") pod \"4a933732-a923-45d2-afc3-e802f0c8cbfa\" (UID: \"4a933732-a923-45d2-afc3-e802f0c8cbfa\") "
Sep 14 08:12:29.521609 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.521253    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-model-cache" (OuterVolumeSpecName: "model-cache") pod "4a933732-a923-45d2-afc3-e802f0c8cbfa" (UID: "4a933732-a923-45d2-afc3-e802f0c8cbfa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:29.521609 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.521276    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-model-cache\") pod \"5020f734-0410-4125-857a-f16ba73d8567\" (UID: \"5020f734-0410-4125-857a-f16ba73d8567\") "
Sep 14 08:12:29.521609 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.521303    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-tmp-dir\") pod \"bd70e9fd-10f3-4b03-be62-7e130ad88022\" (UID: \"bd70e9fd-10f3-4b03-be62-7e130ad88022\") "
Sep 14 08:12:29.521609 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.521367    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4a933732-a923-45d2-afc3-e802f0c8cbfa" (UID: "4a933732-a923-45d2-afc3-e802f0c8cbfa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:29.522056 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.521758    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-home\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.522056 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.521809    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-tmp-dir\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.522056 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.521826    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-model-cache\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.522056 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.521841    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-home\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.522056 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.521853    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-tmp-dir\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.522056 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.521866    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-model-cache\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.522364 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.522093    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bd70e9fd-10f3-4b03-be62-7e130ad88022" (UID: "bd70e9fd-10f3-4b03-be62-7e130ad88022"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:29.522364 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.522259    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-model-cache" (OuterVolumeSpecName: "model-cache") pod "5020f734-0410-4125-857a-f16ba73d8567" (UID: "5020f734-0410-4125-857a-f16ba73d8567"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:29.523728 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.523685    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-dshm" (OuterVolumeSpecName: "dshm") pod "5020f734-0410-4125-857a-f16ba73d8567" (UID: "5020f734-0410-4125-857a-f16ba73d8567"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:29.523862 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.523813    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd70e9fd-10f3-4b03-be62-7e130ad88022-kube-api-access-d98sl" (OuterVolumeSpecName: "kube-api-access-d98sl") pod "bd70e9fd-10f3-4b03-be62-7e130ad88022" (UID: "bd70e9fd-10f3-4b03-be62-7e130ad88022"). InnerVolumeSpecName "kube-api-access-d98sl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:12:29.523950 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.523896    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-home" (OuterVolumeSpecName: "home") pod "bd70e9fd-10f3-4b03-be62-7e130ad88022" (UID: "bd70e9fd-10f3-4b03-be62-7e130ad88022"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:29.524006 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.523958    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a933732-a923-45d2-afc3-e802f0c8cbfa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4a933732-a923-45d2-afc3-e802f0c8cbfa" (UID: "4a933732-a923-45d2-afc3-e802f0c8cbfa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:12:29.525642 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.525605    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd70e9fd-10f3-4b03-be62-7e130ad88022-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bd70e9fd-10f3-4b03-be62-7e130ad88022" (UID: "bd70e9fd-10f3-4b03-be62-7e130ad88022"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:12:29.525642 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.525624    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-dshm" (OuterVolumeSpecName: "dshm") pod "bd70e9fd-10f3-4b03-be62-7e130ad88022" (UID: "bd70e9fd-10f3-4b03-be62-7e130ad88022"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:29.525772 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.525642    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a933732-a923-45d2-afc3-e802f0c8cbfa-kube-api-access-gmzn7" (OuterVolumeSpecName: "kube-api-access-gmzn7") pod "4a933732-a923-45d2-afc3-e802f0c8cbfa" (UID: "4a933732-a923-45d2-afc3-e802f0c8cbfa"). InnerVolumeSpecName "kube-api-access-gmzn7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:12:29.525772 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.525674    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5020f734-0410-4125-857a-f16ba73d8567-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5020f734-0410-4125-857a-f16ba73d8567" (UID: "5020f734-0410-4125-857a-f16ba73d8567"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:12:29.525772 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.525713    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5020f734-0410-4125-857a-f16ba73d8567-kube-api-access-6xmrf" (OuterVolumeSpecName: "kube-api-access-6xmrf") pod "5020f734-0410-4125-857a-f16ba73d8567" (UID: "5020f734-0410-4125-857a-f16ba73d8567"). InnerVolumeSpecName "kube-api-access-6xmrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:12:29.525772 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.525717    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-dshm" (OuterVolumeSpecName: "dshm") pod "4a933732-a923-45d2-afc3-e802f0c8cbfa" (UID: "4a933732-a923-45d2-afc3-e802f0c8cbfa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:29.558262 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.558220    2562 generic.go:356] "Generic (PLEG): container finished" podID="bd70e9fd-10f3-4b03-be62-7e130ad88022" containerID="29c18e1a93f9b8ff81ca6c24fd5f9414a8d602ba4d37a525181b13787f2f7db8" exitCode=0
Sep 14 08:12:29.558438 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.558274    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"
Sep 14 08:12:29.558438 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.558308    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt" event={"ID":"bd70e9fd-10f3-4b03-be62-7e130ad88022","Type":"ContainerDied","Data":"29c18e1a93f9b8ff81ca6c24fd5f9414a8d602ba4d37a525181b13787f2f7db8"}
Sep 14 08:12:29.558438 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.558350    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt" event={"ID":"bd70e9fd-10f3-4b03-be62-7e130ad88022","Type":"ContainerDied","Data":"e6ab544a1c19d873648dc07baa15419dabab0ecd281dfdfe87bb6da56882b43e"}
Sep 14 08:12:29.558438 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.558376    2562 scope.go:117] "RemoveContainer" containerID="29c18e1a93f9b8ff81ca6c24fd5f9414a8d602ba4d37a525181b13787f2f7db8"
Sep 14 08:12:29.559978 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.559953    2562 generic.go:356] "Generic (PLEG): container finished" podID="4a933732-a923-45d2-afc3-e802f0c8cbfa" containerID="871db68cc9d06b0a1e067d3b5cd77ee75df5556b76287c5425e24aeaf97ce1d9" exitCode=0
Sep 14 08:12:29.560105 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.560008    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"
Sep 14 08:12:29.560105 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.560082    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc" event={"ID":"4a933732-a923-45d2-afc3-e802f0c8cbfa","Type":"ContainerDied","Data":"871db68cc9d06b0a1e067d3b5cd77ee75df5556b76287c5425e24aeaf97ce1d9"}
Sep 14 08:12:29.560223 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.560111    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc" event={"ID":"4a933732-a923-45d2-afc3-e802f0c8cbfa","Type":"ContainerDied","Data":"0e97d1e704f815fd2f8e09c5c7d7883ec2b0fdcc8ec3de5d33242c7135584ee9"}
Sep 14 08:12:29.561716 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.561644    2562 generic.go:356] "Generic (PLEG): container finished" podID="5020f734-0410-4125-857a-f16ba73d8567" containerID="be067c52971278fbefa75ca71f889033b41b9d9842f8dd26e0fbdb86768eb46d" exitCode=0
Sep 14 08:12:29.561716 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.561700    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"
Sep 14 08:12:29.561862 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.561694    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq" event={"ID":"5020f734-0410-4125-857a-f16ba73d8567","Type":"ContainerDied","Data":"be067c52971278fbefa75ca71f889033b41b9d9842f8dd26e0fbdb86768eb46d"}
Sep 14 08:12:29.561862 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.561807    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq" event={"ID":"5020f734-0410-4125-857a-f16ba73d8567","Type":"ContainerDied","Data":"47e407f21817906156b793bec9f0c2678a0ecd50421a1ce16dcc5961b7a79adc"}
Sep 14 08:12:29.569252 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.569235    2562 scope.go:117] "RemoveContainer" containerID="29c18e1a93f9b8ff81ca6c24fd5f9414a8d602ba4d37a525181b13787f2f7db8"
Sep 14 08:12:29.569574 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:12:29.569556    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29c18e1a93f9b8ff81ca6c24fd5f9414a8d602ba4d37a525181b13787f2f7db8\": container with ID starting with 29c18e1a93f9b8ff81ca6c24fd5f9414a8d602ba4d37a525181b13787f2f7db8 not found: ID does not exist" containerID="29c18e1a93f9b8ff81ca6c24fd5f9414a8d602ba4d37a525181b13787f2f7db8"
Sep 14 08:12:29.569670 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.569580    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29c18e1a93f9b8ff81ca6c24fd5f9414a8d602ba4d37a525181b13787f2f7db8"} err="failed to get container status \"29c18e1a93f9b8ff81ca6c24fd5f9414a8d602ba4d37a525181b13787f2f7db8\": rpc error: code = NotFound desc = could not find container \"29c18e1a93f9b8ff81ca6c24fd5f9414a8d602ba4d37a525181b13787f2f7db8\": container with ID starting with 29c18e1a93f9b8ff81ca6c24fd5f9414a8d602ba4d37a525181b13787f2f7db8 not found: ID does not exist"
Sep 14 08:12:29.569670 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.569598    2562 scope.go:117] "RemoveContainer" containerID="871db68cc9d06b0a1e067d3b5cd77ee75df5556b76287c5425e24aeaf97ce1d9"
Sep 14 08:12:29.579798 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.579778    2562 scope.go:117] "RemoveContainer" containerID="871db68cc9d06b0a1e067d3b5cd77ee75df5556b76287c5425e24aeaf97ce1d9"
Sep 14 08:12:29.580090 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:12:29.580066    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"871db68cc9d06b0a1e067d3b5cd77ee75df5556b76287c5425e24aeaf97ce1d9\": container with ID starting with 871db68cc9d06b0a1e067d3b5cd77ee75df5556b76287c5425e24aeaf97ce1d9 not found: ID does not exist" containerID="871db68cc9d06b0a1e067d3b5cd77ee75df5556b76287c5425e24aeaf97ce1d9"
Sep 14 08:12:29.580161 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.580096    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"871db68cc9d06b0a1e067d3b5cd77ee75df5556b76287c5425e24aeaf97ce1d9"} err="failed to get container status \"871db68cc9d06b0a1e067d3b5cd77ee75df5556b76287c5425e24aeaf97ce1d9\": rpc error: code = NotFound desc = could not find container \"871db68cc9d06b0a1e067d3b5cd77ee75df5556b76287c5425e24aeaf97ce1d9\": container with ID starting with 871db68cc9d06b0a1e067d3b5cd77ee75df5556b76287c5425e24aeaf97ce1d9 not found: ID does not exist"
Sep 14 08:12:29.580161 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.580112    2562 scope.go:117] "RemoveContainer" containerID="be067c52971278fbefa75ca71f889033b41b9d9842f8dd26e0fbdb86768eb46d"
Sep 14 08:12:29.589003 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.588970    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"]
Sep 14 08:12:29.589227 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.589209    2562 scope.go:117] "RemoveContainer" containerID="be067c52971278fbefa75ca71f889033b41b9d9842f8dd26e0fbdb86768eb46d"
Sep 14 08:12:29.589622 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:12:29.589594    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be067c52971278fbefa75ca71f889033b41b9d9842f8dd26e0fbdb86768eb46d\": container with ID starting with be067c52971278fbefa75ca71f889033b41b9d9842f8dd26e0fbdb86768eb46d not found: ID does not exist" containerID="be067c52971278fbefa75ca71f889033b41b9d9842f8dd26e0fbdb86768eb46d"
Sep 14 08:12:29.589750 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.589632    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be067c52971278fbefa75ca71f889033b41b9d9842f8dd26e0fbdb86768eb46d"} err="failed to get container status \"be067c52971278fbefa75ca71f889033b41b9d9842f8dd26e0fbdb86768eb46d\": rpc error: code = NotFound desc = could not find container \"be067c52971278fbefa75ca71f889033b41b9d9842f8dd26e0fbdb86768eb46d\": container with ID starting with be067c52971278fbefa75ca71f889033b41b9d9842f8dd26e0fbdb86768eb46d not found: ID does not exist"
Sep 14 08:12:29.591990 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.591968    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/simulator-w4-b69a34-kserve-68455bf9c4-c2ktt"]
Sep 14 08:12:29.602112 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.602088    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"]
Sep 14 08:12:29.605752 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.605720    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/distinct-2-w4-b69a34-kserve-774d47fdb9-n4pdc"]
Sep 14 08:12:29.616670 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.616603    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"]
Sep 14 08:12:29.622752 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.622727    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/distinct-w4-b69a34-kserve-567b4c7f98-lfdhq"]
Sep 14 08:12:29.623127 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.623110    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-dshm\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.623175 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.623130    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a933732-a923-45d2-afc3-e802f0c8cbfa-tls-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.623175 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.623139    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-model-cache\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.623175 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.623147    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-tmp-dir\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.623175 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.623156    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xmrf\" (UniqueName: \"kubernetes.io/projected/5020f734-0410-4125-857a-f16ba73d8567-kube-api-access-6xmrf\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.623175 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.623166    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d98sl\" (UniqueName: \"kubernetes.io/projected/bd70e9fd-10f3-4b03-be62-7e130ad88022-kube-api-access-d98sl\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.623331 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.623175    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5020f734-0410-4125-857a-f16ba73d8567-dshm\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.623331 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.623186    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5020f734-0410-4125-857a-f16ba73d8567-tls-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.623331 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.623194    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmzn7\" (UniqueName: \"kubernetes.io/projected/4a933732-a923-45d2-afc3-e802f0c8cbfa-kube-api-access-gmzn7\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.623331 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.623204    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd70e9fd-10f3-4b03-be62-7e130ad88022-tls-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.623331 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.623212    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd70e9fd-10f3-4b03-be62-7e130ad88022-home\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.623331 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:29.623219    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a933732-a923-45d2-afc3-e802f0c8cbfa-dshm\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:30.113932 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:30.113776    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a933732-a923-45d2-afc3-e802f0c8cbfa" path="/var/lib/kubelet/pods/4a933732-a923-45d2-afc3-e802f0c8cbfa/volumes"
Sep 14 08:12:30.114409 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:30.114386    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5020f734-0410-4125-857a-f16ba73d8567" path="/var/lib/kubelet/pods/5020f734-0410-4125-857a-f16ba73d8567/volumes"
Sep 14 08:12:30.114932 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:30.114912    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd70e9fd-10f3-4b03-be62-7e130ad88022" path="/var/lib/kubelet/pods/bd70e9fd-10f3-4b03-be62-7e130ad88022/volumes"
Sep 14 08:12:30.226396 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:30.226357    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:30.964262 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:30.964229    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:31.649233 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:31.649195    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:32.544771 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:32.544732    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:33.250677 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:33.250634    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:34.647403 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:34.647365    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:34.747405 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:34.747371    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:34.911685 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:34.911588    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:38.906249 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:38.906215    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:38.972738 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:38.972705    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:39.034424 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:39.034385    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:39.112829 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:39.112793    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:39.174053 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:39.173956    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:39.486604 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:39.486565    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:39.587074 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:39.587038    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:40.348540 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:40.348490    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:41.095362 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:41.095320    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:42.014337 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:42.014299    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:42.698221 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:42.698184    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:43.396991 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:43.396951    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:45.237239 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:45.237191    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:45.302430 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:45.302395    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:45.482413 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:45.482381    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"]
Sep 14 08:12:45.482634 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:45.482605    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4" podUID="39b6749e-8f5c-460e-bea4-1c0c44b053ae" containerName="istio-proxy" containerID="cri-o://d6f25cbda052875dcee896f642bf6fa5edb3c8046c0189c3e8ed8ea2467f28f8" gracePeriod=30
Sep 14 08:12:45.502361 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:45.502116    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:45.977867 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:45.977829    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:46.500106 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:46.500070    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:12:46.801298 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:46.801204    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:47.602475 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:47.602439    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:48.469993 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:48.469959    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:49.330614 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:49.330582    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:50.098024 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.097992    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:50.614353 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.614320    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:50.676099 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.676064    2562 generic.go:356] "Generic (PLEG): container finished" podID="39b6749e-8f5c-460e-bea4-1c0c44b053ae" containerID="d6f25cbda052875dcee896f642bf6fa5edb3c8046c0189c3e8ed8ea2467f28f8" exitCode=0
Sep 14 08:12:50.676253 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.676098    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4" event={"ID":"39b6749e-8f5c-460e-bea4-1c0c44b053ae","Type":"ContainerDied","Data":"d6f25cbda052875dcee896f642bf6fa5edb3c8046c0189c3e8ed8ea2467f28f8"}
Sep 14 08:12:50.676253 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.676187    2562 scope.go:117] "RemoveContainer" containerID="6e867094c2ec75ae092c24e567e2c07c7533c4f5920241b383a86d7fa4e80e55"
Sep 14 08:12:50.737786 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.737762    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:12:50.838248 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.838151    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m24zq\" (UniqueName: \"kubernetes.io/projected/39b6749e-8f5c-460e-bea4-1c0c44b053ae-kube-api-access-m24zq\") pod \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") "
Sep 14 08:12:50.838417 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.838254    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-data\") pod \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") "
Sep 14 08:12:50.838417 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.838304    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-envoy\") pod \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") "
Sep 14 08:12:50.838417 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.838339    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-credential-socket\") pod \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") "
Sep 14 08:12:50.838417 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.838405    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-podinfo\") pod \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") "
Sep 14 08:12:50.838669 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.838444    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-token\") pod \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") "
Sep 14 08:12:50.838669 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.838474    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-workload-socket\") pod \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") "
Sep 14 08:12:50.838669 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.838520    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-workload-certs\") pod \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") "
Sep 14 08:12:50.838669 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.838561    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-data" (OuterVolumeSpecName: "istio-data") pod "39b6749e-8f5c-460e-bea4-1c0c44b053ae" (UID: "39b6749e-8f5c-460e-bea4-1c0c44b053ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:50.838669 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.838603    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istiod-ca-cert\") pod \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\" (UID: \"39b6749e-8f5c-460e-bea4-1c0c44b053ae\") "
Sep 14 08:12:50.838931 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.838876    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "39b6749e-8f5c-460e-bea4-1c0c44b053ae" (UID: "39b6749e-8f5c-460e-bea4-1c0c44b053ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:50.838931 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.838895    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "39b6749e-8f5c-460e-bea4-1c0c44b053ae" (UID: "39b6749e-8f5c-460e-bea4-1c0c44b053ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:50.838931 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.838906    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "39b6749e-8f5c-460e-bea4-1c0c44b053ae" (UID: "39b6749e-8f5c-460e-bea4-1c0c44b053ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:50.839094 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.838985    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-data\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:50.839094 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.839005    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-credential-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:50.839094 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.839020    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-workload-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:50.839094 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.839034    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-workload-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:50.839307 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.839151    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "39b6749e-8f5c-460e-bea4-1c0c44b053ae" (UID: "39b6749e-8f5c-460e-bea4-1c0c44b053ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:12:50.842229 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.842203    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39b6749e-8f5c-460e-bea4-1c0c44b053ae-kube-api-access-m24zq" (OuterVolumeSpecName: "kube-api-access-m24zq") pod "39b6749e-8f5c-460e-bea4-1c0c44b053ae" (UID: "39b6749e-8f5c-460e-bea4-1c0c44b053ae"). InnerVolumeSpecName "kube-api-access-m24zq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:12:50.843635 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.843601    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "39b6749e-8f5c-460e-bea4-1c0c44b053ae" (UID: "39b6749e-8f5c-460e-bea4-1c0c44b053ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:12:50.843635 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.843599    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "39b6749e-8f5c-460e-bea4-1c0c44b053ae" (UID: "39b6749e-8f5c-460e-bea4-1c0c44b053ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:50.843781 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.843718    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-token" (OuterVolumeSpecName: "istio-token") pod "39b6749e-8f5c-460e-bea4-1c0c44b053ae" (UID: "39b6749e-8f5c-460e-bea4-1c0c44b053ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:12:50.870954 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.870918    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:50.939785 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.939749    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-envoy\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:50.939785 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.939780    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-podinfo\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:50.939785 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.939788    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istio-token\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:50.940019 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.939796    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/39b6749e-8f5c-460e-bea4-1c0c44b053ae-istiod-ca-cert\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:50.940019 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.939805    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m24zq\" (UniqueName: \"kubernetes.io/projected/39b6749e-8f5c-460e-bea4-1c0c44b053ae-kube-api-access-m24zq\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:12:50.969978 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:50.969944    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:51.069889 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:51.069853    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:51.682883 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:51.682853    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"
Sep 14 08:12:51.683371 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:51.682893    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4" event={"ID":"39b6749e-8f5c-460e-bea4-1c0c44b053ae","Type":"ContainerDied","Data":"11d7ce2fd9f0dee06980e038a5adf4a198940450b596bda4606de2d35c0b96a3"}
Sep 14 08:12:51.683371 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:51.682964    2562 scope.go:117] "RemoveContainer" containerID="d6f25cbda052875dcee896f642bf6fa5edb3c8046c0189c3e8ed8ea2467f28f8"
Sep 14 08:12:51.706643 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:51.706614    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"]
Sep 14 08:12:51.710176 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:51.710147    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-b914f7-openshift-default-5545ddd58b-fbtl4"]
Sep 14 08:12:51.764606 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:51.764564    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:51.960242 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:51.960204    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:52.092570 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:52.092540    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:52.112558 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:52.112524    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39b6749e-8f5c-460e-bea4-1c0c44b053ae" path="/var/lib/kubelet/pods/39b6749e-8f5c-460e-bea4-1c0c44b053ae/volumes"
Sep 14 08:12:56.025109 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:56.025075    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:12:57.623863 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:12:57.623825    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:01.617456 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:01.617424    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"]
Sep 14 08:13:01.617937 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:01.617719    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2" podUID="b8fc99a0-16f5-4cfb-ac28-569ec936bcce" containerName="maas-api" containerID="cri-o://d6fdb5920d5794b26f6335a410aac82f76381d62c9358ea6fd2341343c8a7d8c" gracePeriod=30
Sep 14 08:13:01.643033 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:01.643000    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:01.712978 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:01.712927    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:01.743139 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:01.743107    2562 generic.go:356] "Generic (PLEG): container finished" podID="b8fc99a0-16f5-4cfb-ac28-569ec936bcce" containerID="d6fdb5920d5794b26f6335a410aac82f76381d62c9358ea6fd2341343c8a7d8c" exitCode=0
Sep 14 08:13:01.743302 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:01.743181    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2" event={"ID":"b8fc99a0-16f5-4cfb-ac28-569ec936bcce","Type":"ContainerDied","Data":"d6fdb5920d5794b26f6335a410aac82f76381d62c9358ea6fd2341343c8a7d8c"}
Sep 14 08:13:01.766752 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:01.766712    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:01.858317 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:01.858282    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:01.864836 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:01.864811    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"
Sep 14 08:13:01.929958 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:01.929867    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:01.951210 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:01.951176    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-maas-api-tls\") pod \"b8fc99a0-16f5-4cfb-ac28-569ec936bcce\" (UID: \"b8fc99a0-16f5-4cfb-ac28-569ec936bcce\") "
Sep 14 08:13:01.951347 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:01.951239    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxbq6\" (UniqueName: \"kubernetes.io/projected/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-kube-api-access-gxbq6\") pod \"b8fc99a0-16f5-4cfb-ac28-569ec936bcce\" (UID: \"b8fc99a0-16f5-4cfb-ac28-569ec936bcce\") "
Sep 14 08:13:01.951347 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:01.951288    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-metrics-certs\") pod \"b8fc99a0-16f5-4cfb-ac28-569ec936bcce\" (UID: \"b8fc99a0-16f5-4cfb-ac28-569ec936bcce\") "
Sep 14 08:13:01.951347 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:01.951314    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-openshift-service-ca\") pod \"b8fc99a0-16f5-4cfb-ac28-569ec936bcce\" (UID: \"b8fc99a0-16f5-4cfb-ac28-569ec936bcce\") "
Sep 14 08:13:01.951809 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:01.951784    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b8fc99a0-16f5-4cfb-ac28-569ec936bcce" (UID: "b8fc99a0-16f5-4cfb-ac28-569ec936bcce"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:13:01.953455 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:01.953429    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-kube-api-access-gxbq6" (OuterVolumeSpecName: "kube-api-access-gxbq6") pod "b8fc99a0-16f5-4cfb-ac28-569ec936bcce" (UID: "b8fc99a0-16f5-4cfb-ac28-569ec936bcce"). InnerVolumeSpecName "kube-api-access-gxbq6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:13:01.953455 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:01.953450    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b8fc99a0-16f5-4cfb-ac28-569ec936bcce" (UID: "b8fc99a0-16f5-4cfb-ac28-569ec936bcce"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:13:01.953598 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:01.953485    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b8fc99a0-16f5-4cfb-ac28-569ec936bcce" (UID: "b8fc99a0-16f5-4cfb-ac28-569ec936bcce"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:13:02.053134 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:02.053084    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-maas-api-tls\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:02.053134 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:02.053128    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxbq6\" (UniqueName: \"kubernetes.io/projected/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-kube-api-access-gxbq6\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:02.053134 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:02.053139    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-metrics-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:02.053378 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:02.053148    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8fc99a0-16f5-4cfb-ac28-569ec936bcce-openshift-service-ca\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:02.199235 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:02.199199    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:02.321701 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:02.321665    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:02.749919 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:02.749887    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"
Sep 14 08:13:02.750333 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:02.749895    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2" event={"ID":"b8fc99a0-16f5-4cfb-ac28-569ec936bcce","Type":"ContainerDied","Data":"61ef5e86999c2b64d254a68315d673c43397375dc265a7541f3097a0798f84bb"}
Sep 14 08:13:02.750333 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:02.750017    2562 scope.go:117] "RemoveContainer" containerID="d6fdb5920d5794b26f6335a410aac82f76381d62c9358ea6fd2341343c8a7d8c"
Sep 14 08:13:02.768351 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:02.768321    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"]
Sep 14 08:13:02.770031 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:02.770009    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-87b7cb-57d86d644f-rgtx2"]
Sep 14 08:13:03.052251 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:03.052161    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:04.081099 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:04.081058    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:04.112496 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:04.112462    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8fc99a0-16f5-4cfb-ac28-569ec936bcce" path="/var/lib/kubelet/pods/b8fc99a0-16f5-4cfb-ac28-569ec936bcce/volumes"
Sep 14 08:13:04.629320 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:04.629265    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:05.472044 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:05.472009    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:06.104845 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:06.104809    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:13:08.557303 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.554111    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b990f4-xblk9"]
Sep 14 08:13:08.557303 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.555087    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39b6749e-8f5c-460e-bea4-1c0c44b053ae" containerName="istio-proxy"
Sep 14 08:13:08.557303 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.555111    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="39b6749e-8f5c-460e-bea4-1c0c44b053ae" containerName="istio-proxy"
Sep 14 08:13:08.557303 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.555139    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd70e9fd-10f3-4b03-be62-7e130ad88022" containerName="main"
Sep 14 08:13:08.557303 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.555148    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd70e9fd-10f3-4b03-be62-7e130ad88022" containerName="main"
Sep 14 08:13:08.557303 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.555168    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8fc99a0-16f5-4cfb-ac28-569ec936bcce" containerName="maas-api"
Sep 14 08:13:08.557303 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.555177    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8fc99a0-16f5-4cfb-ac28-569ec936bcce" containerName="maas-api"
Sep 14 08:13:08.557303 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.555211    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a933732-a923-45d2-afc3-e802f0c8cbfa" containerName="main"
Sep 14 08:13:08.557303 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.555219    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a933732-a923-45d2-afc3-e802f0c8cbfa" containerName="main"
Sep 14 08:13:08.557303 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.555245    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39b6749e-8f5c-460e-bea4-1c0c44b053ae" containerName="istio-proxy"
Sep 14 08:13:08.557303 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.555259    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="39b6749e-8f5c-460e-bea4-1c0c44b053ae" containerName="istio-proxy"
Sep 14 08:13:08.557303 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.555287    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5020f734-0410-4125-857a-f16ba73d8567" containerName="main"
Sep 14 08:13:08.557303 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.555295    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5020f734-0410-4125-857a-f16ba73d8567" containerName="main"
Sep 14 08:13:08.557303 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.555460    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd70e9fd-10f3-4b03-be62-7e130ad88022" containerName="main"
Sep 14 08:13:08.557303 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.555474    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="39b6749e-8f5c-460e-bea4-1c0c44b053ae" containerName="istio-proxy"
Sep 14 08:13:08.557303 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.555490    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5020f734-0410-4125-857a-f16ba73d8567" containerName="main"
Sep 14 08:13:08.557303 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.555515    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="39b6749e-8f5c-460e-bea4-1c0c44b053ae" containerName="istio-proxy"
Sep 14 08:13:08.557303 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.555522    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8fc99a0-16f5-4cfb-ac28-569ec936bcce" containerName="maas-api"
Sep 14 08:13:08.557303 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.555541    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a933732-a923-45d2-afc3-e802f0c8cbfa" containerName="main"
Sep 14 08:13:08.563691 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.563659    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b990f4-xblk9"
Sep 14 08:13:08.564049 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.564023    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b990f4-xblk9"]
Sep 14 08:13:08.566006 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.565982    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:13:08.566123 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.566047    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:13:08.683885 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.683846    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:08.719211 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.719158    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba607e25-ab58-4f77-9eb5-32d6f74caa7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-b990f4-xblk9\" (UID: \"ba607e25-ab58-4f77-9eb5-32d6f74caa7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b990f4-xblk9"
Sep 14 08:13:08.787396 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.787348    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:08.819796 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.819706    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba607e25-ab58-4f77-9eb5-32d6f74caa7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-b990f4-xblk9\" (UID: \"ba607e25-ab58-4f77-9eb5-32d6f74caa7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b990f4-xblk9"
Sep 14 08:13:08.820363 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.820343    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba607e25-ab58-4f77-9eb5-32d6f74caa7a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-b990f4-xblk9\" (UID: \"ba607e25-ab58-4f77-9eb5-32d6f74caa7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b990f4-xblk9"
Sep 14 08:13:08.874779 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:08.874741    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b990f4-xblk9"
Sep 14 08:13:09.147819 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:09.147787    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b990f4-xblk9"]
Sep 14 08:13:09.150638 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:13:09.150609    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba607e25_ab58_4f77_9eb5_32d6f74caa7a.slice/crio-4876011f8a513e4d7b17fd386b80f830f7c8bcb76579acb5c6cd211ad0e78dc9 WatchSource:0}: Error finding container 4876011f8a513e4d7b17fd386b80f830f7c8bcb76579acb5c6cd211ad0e78dc9: Status 404 returned error can't find the container with id 4876011f8a513e4d7b17fd386b80f830f7c8bcb76579acb5c6cd211ad0e78dc9
Sep 14 08:13:09.563150 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:09.563118    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:09.794163 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:09.794071    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b990f4-xblk9" event={"ID":"ba607e25-ab58-4f77-9eb5-32d6f74caa7a","Type":"ContainerStarted","Data":"6e027226724a85e2a93a2cae971d8e910b7481616fde32e0e9bf61e96523ca5b"}
Sep 14 08:13:09.794163 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:09.794129    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b990f4-xblk9" event={"ID":"ba607e25-ab58-4f77-9eb5-32d6f74caa7a","Type":"ContainerStarted","Data":"4876011f8a513e4d7b17fd386b80f830f7c8bcb76579acb5c6cd211ad0e78dc9"}
Sep 14 08:13:09.810386 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:09.810332    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b990f4-xblk9" podStartSLOduration=1.810317817 podStartE2EDuration="1.810317817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:13:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:13:09.808391094 +0000 UTC m=+1492.264841085" watchObservedRunningTime="2026-09-14 08:13:09.810317817 +0000 UTC m=+1492.266767811"
Sep 14 08:13:10.448691 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:10.448646    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:10.801476 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:10.801382    2562 generic.go:356] "Generic (PLEG): container finished" podID="ba607e25-ab58-4f77-9eb5-32d6f74caa7a" containerID="6e027226724a85e2a93a2cae971d8e910b7481616fde32e0e9bf61e96523ca5b" exitCode=0
Sep 14 08:13:10.801476 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:10.801467    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b990f4-xblk9" event={"ID":"ba607e25-ab58-4f77-9eb5-32d6f74caa7a","Type":"ContainerDied","Data":"6e027226724a85e2a93a2cae971d8e910b7481616fde32e0e9bf61e96523ca5b"}
Sep 14 08:13:11.325999 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:11.325967    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:11.948364 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:11.948337    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b990f4-xblk9"
Sep 14 08:13:12.053174 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:12.053138    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba607e25-ab58-4f77-9eb5-32d6f74caa7a-maas-api-service-ca\") pod \"ba607e25-ab58-4f77-9eb5-32d6f74caa7a\" (UID: \"ba607e25-ab58-4f77-9eb5-32d6f74caa7a\") "
Sep 14 08:13:12.053495 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:12.053469    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba607e25-ab58-4f77-9eb5-32d6f74caa7a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ba607e25-ab58-4f77-9eb5-32d6f74caa7a" (UID: "ba607e25-ab58-4f77-9eb5-32d6f74caa7a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:13:12.056846 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:12.056798    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:12.153911 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:12.153875    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba607e25-ab58-4f77-9eb5-32d6f74caa7a-maas-api-service-ca\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:12.820441 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:12.820409    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b990f4-xblk9"
Sep 14 08:13:12.820658 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:12.820454    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b990f4-xblk9" event={"ID":"ba607e25-ab58-4f77-9eb5-32d6f74caa7a","Type":"ContainerDied","Data":"4876011f8a513e4d7b17fd386b80f830f7c8bcb76579acb5c6cd211ad0e78dc9"}
Sep 14 08:13:12.820658 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:12.820563    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4876011f8a513e4d7b17fd386b80f830f7c8bcb76579acb5c6cd211ad0e78dc9"
Sep 14 08:13:12.841681 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:12.841623    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:14.125053 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:14.125021    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:18.198402 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:18.198366    2562 scope.go:117] "RemoveContainer" containerID="53921550fb615ce9c8d6e2d8a50ca96d987d213a699b1c8ef845c5b4bcf20d93"
Sep 14 08:13:18.829057 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:18.829022    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:19.469958 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:19.469921    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:19.595360 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:19.595330    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:19.674928 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:19.674889    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:19.751942 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:19.751850    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:19.816007 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:19.815971    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b990f4-xblk9"]
Sep 14 08:13:19.818704 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:19.818675    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:19.819571 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:19.819546    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-b990f4-xblk9"]
Sep 14 08:13:19.877202 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:19.877168    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:20.113174 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:20.113090    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba607e25-ab58-4f77-9eb5-32d6f74caa7a" path="/var/lib/kubelet/pods/ba607e25-ab58-4f77-9eb5-32d6f74caa7a/volumes"
Sep 14 08:13:20.174356 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:20.174316    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:20.275277 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:20.275240    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:20.946461 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:20.946419    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:21.713039 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:21.713007    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:22.103984 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:22.103890    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:22.447312 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:22.447281    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:22.805373 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:22.805285    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:13:26.134100 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:26.134058    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:26.197201 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:26.197162    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:26.374909 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:26.374872    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"]
Sep 14 08:13:26.375194 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:26.375168    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp" podUID="934c8412-b086-4b93-87e9-69b2df9826e2" containerName="istio-proxy" containerID="cri-o://04e3b4c076c3dfdd770933c49b6dbeb17669c1eab8317a1b2b53d66b903a43ec" gracePeriod=30
Sep 14 08:13:26.389217 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:26.389116    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:26.716662 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:26.716627    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-afcf8a-ckjt5"]
Sep 14 08:13:26.717054 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:26.717037    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba607e25-ab58-4f77-9eb5-32d6f74caa7a" containerName="revoke-keys"
Sep 14 08:13:26.717111 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:26.717056    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba607e25-ab58-4f77-9eb5-32d6f74caa7a" containerName="revoke-keys"
Sep 14 08:13:26.717147 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:26.717132    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba607e25-ab58-4f77-9eb5-32d6f74caa7a" containerName="revoke-keys"
Sep 14 08:13:26.721924 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:26.721903    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-afcf8a-ckjt5"
Sep 14 08:13:26.724638 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:26.723971    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:13:26.724638 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:26.724016    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:13:26.726803 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:26.726779    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-afcf8a-ckjt5"]
Sep 14 08:13:26.803865 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:26.803827    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8668e3-0310-4834-80db-0c876b38c5ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-afcf8a-ckjt5\" (UID: \"9e8668e3-0310-4834-80db-0c876b38c5ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-afcf8a-ckjt5"
Sep 14 08:13:26.905102 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:26.905069    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8668e3-0310-4834-80db-0c876b38c5ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-afcf8a-ckjt5\" (UID: \"9e8668e3-0310-4834-80db-0c876b38c5ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-afcf8a-ckjt5"
Sep 14 08:13:26.905767 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:26.905746    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8668e3-0310-4834-80db-0c876b38c5ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-afcf8a-ckjt5\" (UID: \"9e8668e3-0310-4834-80db-0c876b38c5ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-afcf8a-ckjt5"
Sep 14 08:13:26.935184 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:26.935148    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:27.034890 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:27.034805    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-afcf8a-ckjt5"
Sep 14 08:13:27.305995 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:27.305965    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-afcf8a-ckjt5"]
Sep 14 08:13:27.308589 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:13:27.308560    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e8668e3_0310_4834_80db_0c876b38c5ad.slice/crio-b66c14e6b66cc633db6308b3d4f9df3e2c27e862f42e9944daee90058eb3ea22 WatchSource:0}: Error finding container b66c14e6b66cc633db6308b3d4f9df3e2c27e862f42e9944daee90058eb3ea22: Status 404 returned error can't find the container with id b66c14e6b66cc633db6308b3d4f9df3e2c27e862f42e9944daee90058eb3ea22
Sep 14 08:13:27.897882 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:27.897796    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-afcf8a-ckjt5" event={"ID":"9e8668e3-0310-4834-80db-0c876b38c5ad","Type":"ContainerStarted","Data":"4140fe56933b97dda88d275c062f289c5d5a1527bd8d5eb438ab6861dc7e8c72"}
Sep 14 08:13:27.897882 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:27.897848    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-afcf8a-ckjt5" event={"ID":"9e8668e3-0310-4834-80db-0c876b38c5ad","Type":"ContainerStarted","Data":"b66c14e6b66cc633db6308b3d4f9df3e2c27e862f42e9944daee90058eb3ea22"}
Sep 14 08:13:27.912779 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:27.912728    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-afcf8a-ckjt5" podStartSLOduration=1.9127132489999998 podStartE2EDuration="1.912713249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:13:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:13:27.910718201 +0000 UTC m=+1510.367168194" watchObservedRunningTime="2026-09-14 08:13:27.912713249 +0000 UTC m=+1510.369163241"
Sep 14 08:13:27.927214 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:27.927182    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:28.313538 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:28.313487    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:28.643346 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:28.643260    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:28.904292 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:28.904200    2562 generic.go:356] "Generic (PLEG): container finished" podID="9e8668e3-0310-4834-80db-0c876b38c5ad" containerID="4140fe56933b97dda88d275c062f289c5d5a1527bd8d5eb438ab6861dc7e8c72" exitCode=0
Sep 14 08:13:28.904445 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:28.904295    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-afcf8a-ckjt5" event={"ID":"9e8668e3-0310-4834-80db-0c876b38c5ad","Type":"ContainerDied","Data":"4140fe56933b97dda88d275c062f289c5d5a1527bd8d5eb438ab6861dc7e8c72"}
Sep 14 08:13:29.363596 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:29.363554    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:30.048293 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:30.048269    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-afcf8a-ckjt5"
Sep 14 08:13:30.140108 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:30.140079    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8668e3-0310-4834-80db-0c876b38c5ad-maas-api-service-ca\") pod \"9e8668e3-0310-4834-80db-0c876b38c5ad\" (UID: \"9e8668e3-0310-4834-80db-0c876b38c5ad\") "
Sep 14 08:13:30.140435 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:30.140409    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e8668e3-0310-4834-80db-0c876b38c5ad-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9e8668e3-0310-4834-80db-0c876b38c5ad" (UID: "9e8668e3-0310-4834-80db-0c876b38c5ad"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:13:30.157177 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:30.157138    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:30.241827 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:30.241790    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9e8668e3-0310-4834-80db-0c876b38c5ad-maas-api-service-ca\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:30.918001 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:30.917970    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-afcf8a-ckjt5"
Sep 14 08:13:30.918377 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:30.918007    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-afcf8a-ckjt5" event={"ID":"9e8668e3-0310-4834-80db-0c876b38c5ad","Type":"ContainerDied","Data":"b66c14e6b66cc633db6308b3d4f9df3e2c27e862f42e9944daee90058eb3ea22"}
Sep 14 08:13:30.918377 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:30.918050    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b66c14e6b66cc633db6308b3d4f9df3e2c27e862f42e9944daee90058eb3ea22"
Sep 14 08:13:31.621046 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.621013    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:13:31.656271 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.656194    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-istio-data\") pod \"934c8412-b086-4b93-87e9-69b2df9826e2\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") "
Sep 14 08:13:31.656271 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.656244    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934c8412-b086-4b93-87e9-69b2df9826e2-istiod-ca-cert\") pod \"934c8412-b086-4b93-87e9-69b2df9826e2\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") "
Sep 14 08:13:31.656494 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.656273    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-credential-socket\") pod \"934c8412-b086-4b93-87e9-69b2df9826e2\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") "
Sep 14 08:13:31.656494 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.656323    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-istio-envoy\") pod \"934c8412-b086-4b93-87e9-69b2df9826e2\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") "
Sep 14 08:13:31.656494 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.656394    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/934c8412-b086-4b93-87e9-69b2df9826e2-istio-token\") pod \"934c8412-b086-4b93-87e9-69b2df9826e2\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") "
Sep 14 08:13:31.656494 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.656427    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdltj\" (UniqueName: \"kubernetes.io/projected/934c8412-b086-4b93-87e9-69b2df9826e2-kube-api-access-bdltj\") pod \"934c8412-b086-4b93-87e9-69b2df9826e2\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") "
Sep 14 08:13:31.656494 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.656482    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934c8412-b086-4b93-87e9-69b2df9826e2-istio-podinfo\") pod \"934c8412-b086-4b93-87e9-69b2df9826e2\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") "
Sep 14 08:13:31.656782 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.656524    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-istio-data" (OuterVolumeSpecName: "istio-data") pod "934c8412-b086-4b93-87e9-69b2df9826e2" (UID: "934c8412-b086-4b93-87e9-69b2df9826e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:31.656782 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.656534    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-workload-socket\") pod \"934c8412-b086-4b93-87e9-69b2df9826e2\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") "
Sep 14 08:13:31.656782 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.656579    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-workload-certs\") pod \"934c8412-b086-4b93-87e9-69b2df9826e2\" (UID: \"934c8412-b086-4b93-87e9-69b2df9826e2\") "
Sep 14 08:13:31.656782 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.656654    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "934c8412-b086-4b93-87e9-69b2df9826e2" (UID: "934c8412-b086-4b93-87e9-69b2df9826e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:31.656782 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.656757    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/934c8412-b086-4b93-87e9-69b2df9826e2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "934c8412-b086-4b93-87e9-69b2df9826e2" (UID: "934c8412-b086-4b93-87e9-69b2df9826e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:13:31.657021 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.656871    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "934c8412-b086-4b93-87e9-69b2df9826e2" (UID: "934c8412-b086-4b93-87e9-69b2df9826e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:31.657021 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.656994    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "934c8412-b086-4b93-87e9-69b2df9826e2" (UID: "934c8412-b086-4b93-87e9-69b2df9826e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:31.657115 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.657095    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-workload-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:31.657182 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.657118    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-workload-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:31.657182 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.657131    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-istio-data\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:31.657182 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.657144    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/934c8412-b086-4b93-87e9-69b2df9826e2-istiod-ca-cert\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:31.657182 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.657156    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-credential-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:31.658674 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.658646    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "934c8412-b086-4b93-87e9-69b2df9826e2" (UID: "934c8412-b086-4b93-87e9-69b2df9826e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:31.658866 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.658845    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/934c8412-b086-4b93-87e9-69b2df9826e2-istio-token" (OuterVolumeSpecName: "istio-token") pod "934c8412-b086-4b93-87e9-69b2df9826e2" (UID: "934c8412-b086-4b93-87e9-69b2df9826e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:13:31.659013 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.658984    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/934c8412-b086-4b93-87e9-69b2df9826e2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "934c8412-b086-4b93-87e9-69b2df9826e2" (UID: "934c8412-b086-4b93-87e9-69b2df9826e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:13:31.659013 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.658996    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/934c8412-b086-4b93-87e9-69b2df9826e2-kube-api-access-bdltj" (OuterVolumeSpecName: "kube-api-access-bdltj") pod "934c8412-b086-4b93-87e9-69b2df9826e2" (UID: "934c8412-b086-4b93-87e9-69b2df9826e2"). InnerVolumeSpecName "kube-api-access-bdltj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:13:31.757871 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.757826    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/934c8412-b086-4b93-87e9-69b2df9826e2-istio-envoy\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:31.757871 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.757862    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/934c8412-b086-4b93-87e9-69b2df9826e2-istio-token\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:31.757871 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.757872    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdltj\" (UniqueName: \"kubernetes.io/projected/934c8412-b086-4b93-87e9-69b2df9826e2-kube-api-access-bdltj\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:31.757871 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.757882    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/934c8412-b086-4b93-87e9-69b2df9826e2-istio-podinfo\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:31.924749 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.924661    2562 generic.go:356] "Generic (PLEG): container finished" podID="934c8412-b086-4b93-87e9-69b2df9826e2" containerID="04e3b4c076c3dfdd770933c49b6dbeb17669c1eab8317a1b2b53d66b903a43ec" exitCode=0
Sep 14 08:13:31.924749 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.924710    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"
Sep 14 08:13:31.925181 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.924749    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp" event={"ID":"934c8412-b086-4b93-87e9-69b2df9826e2","Type":"ContainerDied","Data":"04e3b4c076c3dfdd770933c49b6dbeb17669c1eab8317a1b2b53d66b903a43ec"}
Sep 14 08:13:31.925181 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.924824    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp" event={"ID":"934c8412-b086-4b93-87e9-69b2df9826e2","Type":"ContainerDied","Data":"522544a0a36435bc56c463962b4e39b09ac012dba934d68b96b5033b4e57a453"}
Sep 14 08:13:31.925181 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.924848    2562 scope.go:117] "RemoveContainer" containerID="04e3b4c076c3dfdd770933c49b6dbeb17669c1eab8317a1b2b53d66b903a43ec"
Sep 14 08:13:31.934243 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.934220    2562 scope.go:117] "RemoveContainer" containerID="52850a5bbb296da0e0c5bf4e503eb70afa75e7f5dde55e9a4803290d5fa079c0"
Sep 14 08:13:31.942592 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.942569    2562 scope.go:117] "RemoveContainer" containerID="04e3b4c076c3dfdd770933c49b6dbeb17669c1eab8317a1b2b53d66b903a43ec"
Sep 14 08:13:31.942917 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:13:31.942900    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04e3b4c076c3dfdd770933c49b6dbeb17669c1eab8317a1b2b53d66b903a43ec\": container with ID starting with 04e3b4c076c3dfdd770933c49b6dbeb17669c1eab8317a1b2b53d66b903a43ec not found: ID does not exist" containerID="04e3b4c076c3dfdd770933c49b6dbeb17669c1eab8317a1b2b53d66b903a43ec"
Sep 14 08:13:31.942969 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.942925    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04e3b4c076c3dfdd770933c49b6dbeb17669c1eab8317a1b2b53d66b903a43ec"} err="failed to get container status \"04e3b4c076c3dfdd770933c49b6dbeb17669c1eab8317a1b2b53d66b903a43ec\": rpc error: code = NotFound desc = could not find container \"04e3b4c076c3dfdd770933c49b6dbeb17669c1eab8317a1b2b53d66b903a43ec\": container with ID starting with 04e3b4c076c3dfdd770933c49b6dbeb17669c1eab8317a1b2b53d66b903a43ec not found: ID does not exist"
Sep 14 08:13:31.942969 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.942941    2562 scope.go:117] "RemoveContainer" containerID="52850a5bbb296da0e0c5bf4e503eb70afa75e7f5dde55e9a4803290d5fa079c0"
Sep 14 08:13:31.943176 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:13:31.943161    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52850a5bbb296da0e0c5bf4e503eb70afa75e7f5dde55e9a4803290d5fa079c0\": container with ID starting with 52850a5bbb296da0e0c5bf4e503eb70afa75e7f5dde55e9a4803290d5fa079c0 not found: ID does not exist" containerID="52850a5bbb296da0e0c5bf4e503eb70afa75e7f5dde55e9a4803290d5fa079c0"
Sep 14 08:13:31.943219 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.943180    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52850a5bbb296da0e0c5bf4e503eb70afa75e7f5dde55e9a4803290d5fa079c0"} err="failed to get container status \"52850a5bbb296da0e0c5bf4e503eb70afa75e7f5dde55e9a4803290d5fa079c0\": rpc error: code = NotFound desc = could not find container \"52850a5bbb296da0e0c5bf4e503eb70afa75e7f5dde55e9a4803290d5fa079c0\": container with ID starting with 52850a5bbb296da0e0c5bf4e503eb70afa75e7f5dde55e9a4803290d5fa079c0 not found: ID does not exist"
Sep 14 08:13:31.947726 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.947699    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"]
Sep 14 08:13:31.951375 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:31.951352    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-b990f4-openshift-default-b5488bd4b-jv8zp"]
Sep 14 08:13:32.113339 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:32.113307    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="934c8412-b086-4b93-87e9-69b2df9826e2" path="/var/lib/kubelet/pods/934c8412-b086-4b93-87e9-69b2df9826e2/volumes"
Sep 14 08:13:35.095167 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:35.095131    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:37.240678 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:37.240641    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:37.304173 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:37.304137    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:37.404616 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:37.404581    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:37.459112 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:37.459074    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:37.468080 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:37.468049    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-afcf8a-ckjt5"]
Sep 14 08:13:37.469631 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:37.469610    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-afcf8a-ckjt5"]
Sep 14 08:13:37.845227 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:37.845192    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:37.944117 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:37.944075    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:38.113715 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:38.113630    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e8668e3-0310-4834-80db-0c876b38c5ad" path="/var/lib/kubelet/pods/9e8668e3-0310-4834-80db-0c876b38c5ad/volumes"
Sep 14 08:13:38.753864 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:38.753829    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:39.461555 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:39.461519    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:40.201434 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:40.201396    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:13:43.702619 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:43.702587    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:43.800520 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:43.800470    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:43.937657 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:43.937619    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"]
Sep 14 08:13:43.937912 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:43.937883    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878" podUID="896de534-987b-452c-a3a3-960f67ed3cc8" containerName="istio-proxy" containerID="cri-o://21f9fec4953c939dc7f93d1a482ba06599a1bbbf630d6f9b41832c69bd4a973d" gracePeriod=30
Sep 14 08:13:43.954714 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:43.954638    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:45.011078 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:45.011029    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:45.065895 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:45.065850    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:49.017988 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.017945    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:13:49.193347 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.193318    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:13:49.331483 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.331382    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-istio-data\") pod \"896de534-987b-452c-a3a3-960f67ed3cc8\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") "
Sep 14 08:13:49.331483 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.331433    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-workload-socket\") pod \"896de534-987b-452c-a3a3-960f67ed3cc8\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") "
Sep 14 08:13:49.331483 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.331463    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/896de534-987b-452c-a3a3-960f67ed3cc8-istio-podinfo\") pod \"896de534-987b-452c-a3a3-960f67ed3cc8\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") "
Sep 14 08:13:49.331483 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.331479    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-credential-socket\") pod \"896de534-987b-452c-a3a3-960f67ed3cc8\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") "
Sep 14 08:13:49.331856 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.331548    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-istio-envoy\") pod \"896de534-987b-452c-a3a3-960f67ed3cc8\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") "
Sep 14 08:13:49.331856 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.331596    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5546b\" (UniqueName: \"kubernetes.io/projected/896de534-987b-452c-a3a3-960f67ed3cc8-kube-api-access-5546b\") pod \"896de534-987b-452c-a3a3-960f67ed3cc8\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") "
Sep 14 08:13:49.331856 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.331626    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/896de534-987b-452c-a3a3-960f67ed3cc8-istiod-ca-cert\") pod \"896de534-987b-452c-a3a3-960f67ed3cc8\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") "
Sep 14 08:13:49.331856 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.331656    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-workload-certs\") pod \"896de534-987b-452c-a3a3-960f67ed3cc8\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") "
Sep 14 08:13:49.331856 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.331700    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/896de534-987b-452c-a3a3-960f67ed3cc8-istio-token\") pod \"896de534-987b-452c-a3a3-960f67ed3cc8\" (UID: \"896de534-987b-452c-a3a3-960f67ed3cc8\") "
Sep 14 08:13:49.331856 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.331731    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-istio-data" (OuterVolumeSpecName: "istio-data") pod "896de534-987b-452c-a3a3-960f67ed3cc8" (UID: "896de534-987b-452c-a3a3-960f67ed3cc8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:49.331856 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.331775    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "896de534-987b-452c-a3a3-960f67ed3cc8" (UID: "896de534-987b-452c-a3a3-960f67ed3cc8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:49.331856 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.331811    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "896de534-987b-452c-a3a3-960f67ed3cc8" (UID: "896de534-987b-452c-a3a3-960f67ed3cc8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:49.332256 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.332060    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "896de534-987b-452c-a3a3-960f67ed3cc8" (UID: "896de534-987b-452c-a3a3-960f67ed3cc8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:49.332256 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.332087    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-istio-data\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:49.332256 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.332108    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-workload-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:49.332256 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.332125    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-credential-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:49.332256 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.332105    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/896de534-987b-452c-a3a3-960f67ed3cc8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "896de534-987b-452c-a3a3-960f67ed3cc8" (UID: "896de534-987b-452c-a3a3-960f67ed3cc8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:13:49.333969 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.333937    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/896de534-987b-452c-a3a3-960f67ed3cc8-istio-token" (OuterVolumeSpecName: "istio-token") pod "896de534-987b-452c-a3a3-960f67ed3cc8" (UID: "896de534-987b-452c-a3a3-960f67ed3cc8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:13:49.333969 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.333965    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/896de534-987b-452c-a3a3-960f67ed3cc8-kube-api-access-5546b" (OuterVolumeSpecName: "kube-api-access-5546b") pod "896de534-987b-452c-a3a3-960f67ed3cc8" (UID: "896de534-987b-452c-a3a3-960f67ed3cc8"). InnerVolumeSpecName "kube-api-access-5546b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:13:49.334217 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.334196    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "896de534-987b-452c-a3a3-960f67ed3cc8" (UID: "896de534-987b-452c-a3a3-960f67ed3cc8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:49.334217 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.334205    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/896de534-987b-452c-a3a3-960f67ed3cc8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "896de534-987b-452c-a3a3-960f67ed3cc8" (UID: "896de534-987b-452c-a3a3-960f67ed3cc8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:13:49.432664 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.432626    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/896de534-987b-452c-a3a3-960f67ed3cc8-istio-podinfo\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:49.432664 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.432657    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-istio-envoy\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:49.432664 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.432667    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5546b\" (UniqueName: \"kubernetes.io/projected/896de534-987b-452c-a3a3-960f67ed3cc8-kube-api-access-5546b\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:49.432907 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.432679    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/896de534-987b-452c-a3a3-960f67ed3cc8-istiod-ca-cert\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:49.432907 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.432688    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/896de534-987b-452c-a3a3-960f67ed3cc8-workload-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:49.432907 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:49.432695    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/896de534-987b-452c-a3a3-960f67ed3cc8-istio-token\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:13:50.020202 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:50.020169    2562 generic.go:356] "Generic (PLEG): container finished" podID="896de534-987b-452c-a3a3-960f67ed3cc8" containerID="21f9fec4953c939dc7f93d1a482ba06599a1bbbf630d6f9b41832c69bd4a973d" exitCode=0
Sep 14 08:13:50.020637 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:50.020218    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"
Sep 14 08:13:50.020637 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:50.020255    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878" event={"ID":"896de534-987b-452c-a3a3-960f67ed3cc8","Type":"ContainerDied","Data":"21f9fec4953c939dc7f93d1a482ba06599a1bbbf630d6f9b41832c69bd4a973d"}
Sep 14 08:13:50.020637 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:50.020308    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878" event={"ID":"896de534-987b-452c-a3a3-960f67ed3cc8","Type":"ContainerDied","Data":"ac0b7908e8d8dc052fe238bddf1b19b1b9323f71219dd818526aea3b383ba34b"}
Sep 14 08:13:50.020637 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:50.020330    2562 scope.go:117] "RemoveContainer" containerID="21f9fec4953c939dc7f93d1a482ba06599a1bbbf630d6f9b41832c69bd4a973d"
Sep 14 08:13:50.030270 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:50.030253    2562 scope.go:117] "RemoveContainer" containerID="331fea5c716ada12f03eaee4b5f0fb7a548b2ad03659ae20b54f2670df9e70db"
Sep 14 08:13:50.038633 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:50.038612    2562 scope.go:117] "RemoveContainer" containerID="21f9fec4953c939dc7f93d1a482ba06599a1bbbf630d6f9b41832c69bd4a973d"
Sep 14 08:13:50.038900 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:13:50.038881    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21f9fec4953c939dc7f93d1a482ba06599a1bbbf630d6f9b41832c69bd4a973d\": container with ID starting with 21f9fec4953c939dc7f93d1a482ba06599a1bbbf630d6f9b41832c69bd4a973d not found: ID does not exist" containerID="21f9fec4953c939dc7f93d1a482ba06599a1bbbf630d6f9b41832c69bd4a973d"
Sep 14 08:13:50.038946 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:50.038910    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21f9fec4953c939dc7f93d1a482ba06599a1bbbf630d6f9b41832c69bd4a973d"} err="failed to get container status \"21f9fec4953c939dc7f93d1a482ba06599a1bbbf630d6f9b41832c69bd4a973d\": rpc error: code = NotFound desc = could not find container \"21f9fec4953c939dc7f93d1a482ba06599a1bbbf630d6f9b41832c69bd4a973d\": container with ID starting with 21f9fec4953c939dc7f93d1a482ba06599a1bbbf630d6f9b41832c69bd4a973d not found: ID does not exist"
Sep 14 08:13:50.038946 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:50.038928    2562 scope.go:117] "RemoveContainer" containerID="331fea5c716ada12f03eaee4b5f0fb7a548b2ad03659ae20b54f2670df9e70db"
Sep 14 08:13:50.042306 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:13:50.042272    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"331fea5c716ada12f03eaee4b5f0fb7a548b2ad03659ae20b54f2670df9e70db\": container with ID starting with 331fea5c716ada12f03eaee4b5f0fb7a548b2ad03659ae20b54f2670df9e70db not found: ID does not exist" containerID="331fea5c716ada12f03eaee4b5f0fb7a548b2ad03659ae20b54f2670df9e70db"
Sep 14 08:13:50.042445 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:50.042308    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"331fea5c716ada12f03eaee4b5f0fb7a548b2ad03659ae20b54f2670df9e70db"} err="failed to get container status \"331fea5c716ada12f03eaee4b5f0fb7a548b2ad03659ae20b54f2670df9e70db\": rpc error: code = NotFound desc = could not find container \"331fea5c716ada12f03eaee4b5f0fb7a548b2ad03659ae20b54f2670df9e70db\": container with ID starting with 331fea5c716ada12f03eaee4b5f0fb7a548b2ad03659ae20b54f2670df9e70db not found: ID does not exist"
Sep 14 08:13:50.044975 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:50.044948    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"]
Sep 14 08:13:50.051391 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:50.050207    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-afcf8a-openshift-default-ff5ff7f74-pr878"]
Sep 14 08:13:50.112533 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:13:50.112481    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="896de534-987b-452c-a3a3-960f67ed3cc8" path="/var/lib/kubelet/pods/896de534-987b-452c-a3a3-960f67ed3cc8/volumes"
Sep 14 08:14:11.043240 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:14:11.043202    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:14.699721 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:14:14.699686    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:14:30.303590 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:14:30.303543    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:14:44.601858 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:14:44.601818    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:14:53.490995 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:14:53.490962    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:54.435866 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:14:54.435831    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:14:57.698760 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:14:57.698715    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:15:00.138121 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.138083    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822895-6965l"]
Sep 14 08:15:00.138665 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.138644    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="896de534-987b-452c-a3a3-960f67ed3cc8" containerName="istio-proxy"
Sep 14 08:15:00.138753 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.138667    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="896de534-987b-452c-a3a3-960f67ed3cc8" containerName="istio-proxy"
Sep 14 08:15:00.138753 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.138679    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e8668e3-0310-4834-80db-0c876b38c5ad" containerName="revoke-keys"
Sep 14 08:15:00.138753 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.138686    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e8668e3-0310-4834-80db-0c876b38c5ad" containerName="revoke-keys"
Sep 14 08:15:00.138753 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.138709    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="934c8412-b086-4b93-87e9-69b2df9826e2" containerName="istio-proxy"
Sep 14 08:15:00.138753 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.138719    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="934c8412-b086-4b93-87e9-69b2df9826e2" containerName="istio-proxy"
Sep 14 08:15:00.138753 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.138750    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="896de534-987b-452c-a3a3-960f67ed3cc8" containerName="istio-proxy"
Sep 14 08:15:00.139066 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.138759    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="896de534-987b-452c-a3a3-960f67ed3cc8" containerName="istio-proxy"
Sep 14 08:15:00.139066 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.138776    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="934c8412-b086-4b93-87e9-69b2df9826e2" containerName="istio-proxy"
Sep 14 08:15:00.139066 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.138784    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="934c8412-b086-4b93-87e9-69b2df9826e2" containerName="istio-proxy"
Sep 14 08:15:00.139066 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.138869    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="934c8412-b086-4b93-87e9-69b2df9826e2" containerName="istio-proxy"
Sep 14 08:15:00.139066 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.138882    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="896de534-987b-452c-a3a3-960f67ed3cc8" containerName="istio-proxy"
Sep 14 08:15:00.139066 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.138890    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="896de534-987b-452c-a3a3-960f67ed3cc8" containerName="istio-proxy"
Sep 14 08:15:00.139066 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.138899    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e8668e3-0310-4834-80db-0c876b38c5ad" containerName="revoke-keys"
Sep 14 08:15:00.139066 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.138908    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="934c8412-b086-4b93-87e9-69b2df9826e2" containerName="istio-proxy"
Sep 14 08:15:00.141795 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.141772    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7746e4-29822895-dskdr"]
Sep 14 08:15:00.141920 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.141891    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822895-6965l"
Sep 14 08:15:00.143747 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.143725    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:15:00.145175 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.145147    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7746e4-29822895-dskdr"
Sep 14 08:15:00.147673 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.147643    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822895-6965l"]
Sep 14 08:15:00.151270 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.151120    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7746e4-29822895-dskdr"]
Sep 14 08:15:00.156541 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.156518    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822895-6965l"
Sep 14 08:15:00.163622 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.163598    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7746e4-29822895-dskdr"
Sep 14 08:15:00.435388 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.435362    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29822895-6965l"]
Sep 14 08:15:00.438347 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:15:00.438321    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5986f5dd_78bb_4bcf_968d_d0ff466c4443.slice/crio-30f2f83f14cc40127e68e79a44ac7da5a9cbed1bf577a1e7ea3208704cdbfc6c WatchSource:0}: Error finding container 30f2f83f14cc40127e68e79a44ac7da5a9cbed1bf577a1e7ea3208704cdbfc6c: Status 404 returned error can't find the container with id 30f2f83f14cc40127e68e79a44ac7da5a9cbed1bf577a1e7ea3208704cdbfc6c
Sep 14 08:15:00.441697 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:00.441678    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7746e4-29822895-dskdr"]
Sep 14 08:15:00.444853 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:15:00.444811    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67a60a44_f359_49c3_b88a_18a733262075.slice/crio-01135effa12e88e28cbe450d0a11ef57b09a2673d8292a4e09d7064d8acb0c2b WatchSource:0}: Error finding container 01135effa12e88e28cbe450d0a11ef57b09a2673d8292a4e09d7064d8acb0c2b: Status 404 returned error can't find the container with id 01135effa12e88e28cbe450d0a11ef57b09a2673d8292a4e09d7064d8acb0c2b
Sep 14 08:15:01.384042 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:01.384008    2562 generic.go:356] "Generic (PLEG): container finished" podID="67a60a44-f359-49c3-b88a-18a733262075" containerID="d605e9684839241b5d5a2e0a346a4bc16515ae9370e1165921cf0a09c8f0acc9" exitCode=0
Sep 14 08:15:01.384526 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:01.384099    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7746e4-29822895-dskdr" event={"ID":"67a60a44-f359-49c3-b88a-18a733262075","Type":"ContainerDied","Data":"d605e9684839241b5d5a2e0a346a4bc16515ae9370e1165921cf0a09c8f0acc9"}
Sep 14 08:15:01.384526 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:01.384165    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7746e4-29822895-dskdr" event={"ID":"67a60a44-f359-49c3-b88a-18a733262075","Type":"ContainerStarted","Data":"01135effa12e88e28cbe450d0a11ef57b09a2673d8292a4e09d7064d8acb0c2b"}
Sep 14 08:15:01.386039 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:01.386001    2562 generic.go:356] "Generic (PLEG): container finished" podID="5986f5dd-78bb-4bcf-968d-d0ff466c4443" containerID="499270f914953e51d6564d4c8e176e7611f0df750a66cdccdbeeafa66dd45179" exitCode=0
Sep 14 08:15:01.386157 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:01.386087    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822895-6965l" event={"ID":"5986f5dd-78bb-4bcf-968d-d0ff466c4443","Type":"ContainerDied","Data":"499270f914953e51d6564d4c8e176e7611f0df750a66cdccdbeeafa66dd45179"}
Sep 14 08:15:01.386157 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:01.386135    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822895-6965l" event={"ID":"5986f5dd-78bb-4bcf-968d-d0ff466c4443","Type":"ContainerStarted","Data":"30f2f83f14cc40127e68e79a44ac7da5a9cbed1bf577a1e7ea3208704cdbfc6c"}
Sep 14 08:15:01.428890 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:01.428846    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:02.518808 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:02.518777    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822895-6965l"
Sep 14 08:15:02.555057 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:02.555023    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7746e4-29822895-dskdr"
Sep 14 08:15:03.397076 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:03.397045    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7746e4-29822895-dskdr"
Sep 14 08:15:03.397244 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:03.397047    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7746e4-29822895-dskdr" event={"ID":"67a60a44-f359-49c3-b88a-18a733262075","Type":"ContainerDied","Data":"01135effa12e88e28cbe450d0a11ef57b09a2673d8292a4e09d7064d8acb0c2b"}
Sep 14 08:15:03.397244 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:03.397204    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="01135effa12e88e28cbe450d0a11ef57b09a2673d8292a4e09d7064d8acb0c2b"
Sep 14 08:15:03.398430 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:03.398411    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822895-6965l"
Sep 14 08:15:03.398575 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:03.398459    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29822895-6965l" event={"ID":"5986f5dd-78bb-4bcf-968d-d0ff466c4443","Type":"ContainerDied","Data":"30f2f83f14cc40127e68e79a44ac7da5a9cbed1bf577a1e7ea3208704cdbfc6c"}
Sep 14 08:15:03.398575 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:03.398533    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="30f2f83f14cc40127e68e79a44ac7da5a9cbed1bf577a1e7ea3208704cdbfc6c"
Sep 14 08:15:11.000679 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:11.000599    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:15:42.633754 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:42.633719    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:15:46.104892 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:15:46.104848    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:16:02.601592 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:02.601556    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:16:15.205395 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:15.205339    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:15.307257 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:15.307218    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:17.303207 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:17.303173    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:16:18.259156 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:18.259122    2562 scope.go:117] "RemoveContainer" containerID="65747a3d7aaaa22115b7bba3c37d7d9492d00274cadc52348cf4554ee43dbd48"
Sep 14 08:16:21.854886 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:21.854853    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:22.793749 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:22.793712    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:23.232945 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:23.232909    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:23.555124 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:23.555030    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:23.969594 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:23.969558    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:26.122257 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:26.122222    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:26.308936 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:26.308899    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:26.416056 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:26.415969    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:26.833523 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:26.833479    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:16:27.051265 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.051232    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:27.745091 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.745060    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"]
Sep 14 08:16:27.745538 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.745493    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67a60a44-f359-49c3-b88a-18a733262075" containerName="cleanup"
Sep 14 08:16:27.745593 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.745542    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="67a60a44-f359-49c3-b88a-18a733262075" containerName="cleanup"
Sep 14 08:16:27.745593 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.745567    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5986f5dd-78bb-4bcf-968d-d0ff466c4443" containerName="cleanup"
Sep 14 08:16:27.745593 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.745576    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="5986f5dd-78bb-4bcf-968d-d0ff466c4443" containerName="cleanup"
Sep 14 08:16:27.745702 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.745682    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="67a60a44-f359-49c3-b88a-18a733262075" containerName="cleanup"
Sep 14 08:16:27.745702 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.745696    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="5986f5dd-78bb-4bcf-968d-d0ff466c4443" containerName="cleanup"
Sep 14 08:16:27.753813 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.753776    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"
Sep 14 08:16:27.757967 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.756743    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 08:16:27.757967 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.757054    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"]
Sep 14 08:16:27.757967 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.757521    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:16:27.757967 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.757738    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-g2t2q\""
Sep 14 08:16:27.757967 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.757880    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-7f1623-serving-cert\""
Sep 14 08:16:27.758416 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.758398    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 08:16:27.780416 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.780385    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:27.826776 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.826738    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz\" (UID: \"3cf481b3-bae6-4b5b-93aa-8a90f578cef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"
Sep 14 08:16:27.826776 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.826777    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz\" (UID: \"3cf481b3-bae6-4b5b-93aa-8a90f578cef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"
Sep 14 08:16:27.827030 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.826799    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67glr\" (UniqueName: \"kubernetes.io/projected/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-kube-api-access-67glr\") pod \"maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz\" (UID: \"3cf481b3-bae6-4b5b-93aa-8a90f578cef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"
Sep 14 08:16:27.827030 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.826824    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-metrics-certs\") pod \"maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz\" (UID: \"3cf481b3-bae6-4b5b-93aa-8a90f578cef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"
Sep 14 08:16:27.881302 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.881257    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:27.927892 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.927856    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz\" (UID: \"3cf481b3-bae6-4b5b-93aa-8a90f578cef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"
Sep 14 08:16:27.927892 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.927904    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz\" (UID: \"3cf481b3-bae6-4b5b-93aa-8a90f578cef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"
Sep 14 08:16:27.928157 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.927962    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67glr\" (UniqueName: \"kubernetes.io/projected/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-kube-api-access-67glr\") pod \"maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz\" (UID: \"3cf481b3-bae6-4b5b-93aa-8a90f578cef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"
Sep 14 08:16:27.928157 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:16:27.928005    2562 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-serving-cert: secret "maas-api-e2e-ipp-a-7f1623-serving-cert" not found
Sep 14 08:16:27.928157 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.928018    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-metrics-certs\") pod \"maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz\" (UID: \"3cf481b3-bae6-4b5b-93aa-8a90f578cef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"
Sep 14 08:16:27.928157 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:16:27.928070    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-maas-api-tls podName:3cf481b3-bae6-4b5b-93aa-8a90f578cef8 nodeName:}" failed. No retries permitted until 2026-09-14 08:16:28.428052607 +0000 UTC m=+1690.884502578 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-maas-api-tls") pod "maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz" (UID: "3cf481b3-bae6-4b5b-93aa-8a90f578cef8") : secret "maas-api-e2e-ipp-a-7f1623-serving-cert" not found
Sep 14 08:16:27.928769 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.928751    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz\" (UID: \"3cf481b3-bae6-4b5b-93aa-8a90f578cef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"
Sep 14 08:16:27.930583 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.930557    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-metrics-certs\") pod \"maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz\" (UID: \"3cf481b3-bae6-4b5b-93aa-8a90f578cef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"
Sep 14 08:16:27.935781 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.935758    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67glr\" (UniqueName: \"kubernetes.io/projected/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-kube-api-access-67glr\") pod \"maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz\" (UID: \"3cf481b3-bae6-4b5b-93aa-8a90f578cef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"
Sep 14 08:16:27.998421 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:27.998338    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:28.375198 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:28.375102    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:28.431306 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:28.431273    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz\" (UID: \"3cf481b3-bae6-4b5b-93aa-8a90f578cef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"
Sep 14 08:16:28.433620 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:28.433599    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-maas-api-tls\") pod \"maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz\" (UID: \"3cf481b3-bae6-4b5b-93aa-8a90f578cef8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"
Sep 14 08:16:28.472648 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:28.472603    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:28.667745 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:28.667650    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"
Sep 14 08:16:28.942382 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:28.942354    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"]
Sep 14 08:16:28.944923 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:16:28.944889    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cf481b3_bae6_4b5b_93aa_8a90f578cef8.slice/crio-86c8c6ffd9f0aa739bfd3c8a6a0abaeaa3a95288587a2219bdbe9a357a7f6f4b WatchSource:0}: Error finding container 86c8c6ffd9f0aa739bfd3c8a6a0abaeaa3a95288587a2219bdbe9a357a7f6f4b: Status 404 returned error can't find the container with id 86c8c6ffd9f0aa739bfd3c8a6a0abaeaa3a95288587a2219bdbe9a357a7f6f4b
Sep 14 08:16:29.850558 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:29.850447    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz" event={"ID":"3cf481b3-bae6-4b5b-93aa-8a90f578cef8","Type":"ContainerStarted","Data":"e04a58fddd2818e233526e775cf39db1457da46cc74188f28a6ef9ddc74a7d84"}
Sep 14 08:16:29.850558 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:29.850532    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"
Sep 14 08:16:29.850558 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:29.850553    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz" event={"ID":"3cf481b3-bae6-4b5b-93aa-8a90f578cef8","Type":"ContainerStarted","Data":"86c8c6ffd9f0aa739bfd3c8a6a0abaeaa3a95288587a2219bdbe9a357a7f6f4b"}
Sep 14 08:16:29.868021 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:29.867961    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz" podStartSLOduration=2.285472991 podStartE2EDuration="2.86794223s" totalImagesPullingTime="582.469239ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:16:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:16:29.866105403 +0000 UTC m=+1692.322555397" watchObservedRunningTime="2026-09-14 08:16:29.86794223 +0000 UTC m=+1692.324392214"
Sep 14 08:16:35.860488 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:35.860453    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"
Sep 14 08:16:36.496967 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:36.496931    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:36.740446 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:36.740405    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:37.303215 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:37.303175    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:16:49.865922 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:49.865884    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:50.823695 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:50.823655    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:51.318462 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:51.318416    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:51.534841 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:51.534801    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:52.010197 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:52.010162    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:52.261027 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:52.260935    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:52.336285 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:52.336250    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:52.402366 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:52.402328    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:54.763191 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:54.763155    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:55.502163 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:55.502110    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:55.604542 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:55.604492    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:55.702106 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:55.702073    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:56.520276 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:56.520242    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:57.447036 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:57.446999    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:58.213114 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.213075    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"]
Sep 14 08:16:58.217287 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.217265    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"
Sep 14 08:16:58.219236 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.219215    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-56708c-serving-cert\""
Sep 14 08:16:58.227452 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.226946    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"]
Sep 14 08:16:58.251431 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.251391    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:58.318257 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.318224    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2387d0f-4dba-412b-99f3-60591216d4e9-maas-api-tls\") pod \"maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf\" (UID: \"c2387d0f-4dba-412b-99f3-60591216d4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"
Sep 14 08:16:58.318419 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.318292    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2387d0f-4dba-412b-99f3-60591216d4e9-metrics-certs\") pod \"maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf\" (UID: \"c2387d0f-4dba-412b-99f3-60591216d4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"
Sep 14 08:16:58.318419 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.318375    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2387d0f-4dba-412b-99f3-60591216d4e9-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf\" (UID: \"c2387d0f-4dba-412b-99f3-60591216d4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"
Sep 14 08:16:58.318494 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.318430    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwlrh\" (UniqueName: \"kubernetes.io/projected/c2387d0f-4dba-412b-99f3-60591216d4e9-kube-api-access-kwlrh\") pod \"maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf\" (UID: \"c2387d0f-4dba-412b-99f3-60591216d4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"
Sep 14 08:16:58.353350 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.353315    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:58.419032 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.418997    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2387d0f-4dba-412b-99f3-60591216d4e9-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf\" (UID: \"c2387d0f-4dba-412b-99f3-60591216d4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"
Sep 14 08:16:58.419032 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.419040    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwlrh\" (UniqueName: \"kubernetes.io/projected/c2387d0f-4dba-412b-99f3-60591216d4e9-kube-api-access-kwlrh\") pod \"maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf\" (UID: \"c2387d0f-4dba-412b-99f3-60591216d4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"
Sep 14 08:16:58.419289 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.419114    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2387d0f-4dba-412b-99f3-60591216d4e9-maas-api-tls\") pod \"maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf\" (UID: \"c2387d0f-4dba-412b-99f3-60591216d4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"
Sep 14 08:16:58.419289 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.419170    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2387d0f-4dba-412b-99f3-60591216d4e9-metrics-certs\") pod \"maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf\" (UID: \"c2387d0f-4dba-412b-99f3-60591216d4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"
Sep 14 08:16:58.419822 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.419794    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2387d0f-4dba-412b-99f3-60591216d4e9-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf\" (UID: \"c2387d0f-4dba-412b-99f3-60591216d4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"
Sep 14 08:16:58.422008 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.421981    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2387d0f-4dba-412b-99f3-60591216d4e9-metrics-certs\") pod \"maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf\" (UID: \"c2387d0f-4dba-412b-99f3-60591216d4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"
Sep 14 08:16:58.422117 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.422027    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2387d0f-4dba-412b-99f3-60591216d4e9-maas-api-tls\") pod \"maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf\" (UID: \"c2387d0f-4dba-412b-99f3-60591216d4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"
Sep 14 08:16:58.427355 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.427331    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwlrh\" (UniqueName: \"kubernetes.io/projected/c2387d0f-4dba-412b-99f3-60591216d4e9-kube-api-access-kwlrh\") pod \"maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf\" (UID: \"c2387d0f-4dba-412b-99f3-60591216d4e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"
Sep 14 08:16:58.452873 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.452833    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:58.533096 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.533009    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"
Sep 14 08:16:58.731685 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.731649    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:58.804153 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.804121    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"]
Sep 14 08:16:58.806258 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:58.806228    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:16:58.806920 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:16:58.806891    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2387d0f_4dba_412b_99f3_60591216d4e9.slice/crio-f7892030fe6cb720f1a8828f3c611eb988623dcb70a94285a94a73dcfd848d44 WatchSource:0}: Error finding container f7892030fe6cb720f1a8828f3c611eb988623dcb70a94285a94a73dcfd848d44: Status 404 returned error can't find the container with id f7892030fe6cb720f1a8828f3c611eb988623dcb70a94285a94a73dcfd848d44
Sep 14 08:16:59.005306 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:16:59.005266    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf" event={"ID":"c2387d0f-4dba-412b-99f3-60591216d4e9","Type":"ContainerStarted","Data":"f7892030fe6cb720f1a8828f3c611eb988623dcb70a94285a94a73dcfd848d44"}
Sep 14 08:17:00.011727 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:00.011671    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf" event={"ID":"c2387d0f-4dba-412b-99f3-60591216d4e9","Type":"ContainerStarted","Data":"3f69d25425b3b96b466e3eb9a4e633b469cc49a629b8643544197b05c7713b63"}
Sep 14 08:17:00.012148 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:00.011809    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"
Sep 14 08:17:00.028719 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:00.028666    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf" podStartSLOduration=1.6829579159999999 podStartE2EDuration="2.028651822s" totalImagesPullingTime="345.693906ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:16:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:17:00.02663579 +0000 UTC m=+1722.483085785" watchObservedRunningTime="2026-09-14 08:17:00.028651822 +0000 UTC m=+1722.485101814"
Sep 14 08:17:02.998854 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:02.998818    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:17:06.022181 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:06.022148    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"
Sep 14 08:17:20.279988 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:20.279948    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:17:20.484201 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:20.484150    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:17:21.211896 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:21.211862    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:17:21.925329 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:21.925293    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:17:22.671461 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:22.671417    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:17:23.423303 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:23.423271    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:17:34.493935 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.493892    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"]
Sep 14 08:17:34.498387 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.498363    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:34.501291 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.501264    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-7f1623\"/\"default-dockercfg-kb7dc\""
Sep 14 08:17:34.501757 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.501626    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-7f1623\"/\"ipp-route-7f1623-kserve-self-signed-certs\""
Sep 14 08:17:34.501912 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.501881    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-7f1623\"/\"openshift-service-ca.crt\""
Sep 14 08:17:34.502160 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.502139    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-7f1623\"/\"kube-root-ca.crt\""
Sep 14 08:17:34.504143 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.504118    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"]
Sep 14 08:17:34.525531 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.525488    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:17:34.559618 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.559589    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-tmp-dir\") pod \"ipp-route-7f1623-kserve-668bcc7b95-tg266\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") " pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:34.559775 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.559627    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a73bb590-a5a9-4094-8e4e-0d235f10752e-tls-certs\") pod \"ipp-route-7f1623-kserve-668bcc7b95-tg266\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") " pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:34.559775 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.559695    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-dshm\") pod \"ipp-route-7f1623-kserve-668bcc7b95-tg266\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") " pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:34.559775 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.559767    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-model-cache\") pod \"ipp-route-7f1623-kserve-668bcc7b95-tg266\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") " pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:34.559945 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.559823    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-home\") pod \"ipp-route-7f1623-kserve-668bcc7b95-tg266\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") " pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:34.559945 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.559867    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5t2d\" (UniqueName: \"kubernetes.io/projected/a73bb590-a5a9-4094-8e4e-0d235f10752e-kube-api-access-m5t2d\") pod \"ipp-route-7f1623-kserve-668bcc7b95-tg266\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") " pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:34.661289 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.661256    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-home\") pod \"ipp-route-7f1623-kserve-668bcc7b95-tg266\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") " pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:34.661289 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.661301    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5t2d\" (UniqueName: \"kubernetes.io/projected/a73bb590-a5a9-4094-8e4e-0d235f10752e-kube-api-access-m5t2d\") pod \"ipp-route-7f1623-kserve-668bcc7b95-tg266\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") " pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:34.661581 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.661341    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-tmp-dir\") pod \"ipp-route-7f1623-kserve-668bcc7b95-tg266\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") " pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:34.661581 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.661359    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a73bb590-a5a9-4094-8e4e-0d235f10752e-tls-certs\") pod \"ipp-route-7f1623-kserve-668bcc7b95-tg266\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") " pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:34.661581 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.661387    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-dshm\") pod \"ipp-route-7f1623-kserve-668bcc7b95-tg266\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") " pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:34.661581 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.661532    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-model-cache\") pod \"ipp-route-7f1623-kserve-668bcc7b95-tg266\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") " pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:34.661776 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.661705    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-home\") pod \"ipp-route-7f1623-kserve-668bcc7b95-tg266\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") " pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:34.661814 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.661767    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-tmp-dir\") pod \"ipp-route-7f1623-kserve-668bcc7b95-tg266\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") " pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:34.661881 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.661865    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-model-cache\") pod \"ipp-route-7f1623-kserve-668bcc7b95-tg266\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") " pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:34.663577 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.663551    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-dshm\") pod \"ipp-route-7f1623-kserve-668bcc7b95-tg266\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") " pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:34.663912 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.663893    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a73bb590-a5a9-4094-8e4e-0d235f10752e-tls-certs\") pod \"ipp-route-7f1623-kserve-668bcc7b95-tg266\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") " pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:34.668929 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.668907    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5t2d\" (UniqueName: \"kubernetes.io/projected/a73bb590-a5a9-4094-8e4e-0d235f10752e-kube-api-access-m5t2d\") pod \"ipp-route-7f1623-kserve-668bcc7b95-tg266\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") " pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:34.809928 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:34.809829    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:35.080696 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:35.080668    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"]
Sep 14 08:17:35.082760 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:17:35.082724    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda73bb590_a5a9_4094_8e4e_0d235f10752e.slice/crio-63d78a8ece16785b2b36df6d270c8b5d7e3ece0282664076c7b456b052fe0fd1 WatchSource:0}: Error finding container 63d78a8ece16785b2b36df6d270c8b5d7e3ece0282664076c7b456b052fe0fd1: Status 404 returned error can't find the container with id 63d78a8ece16785b2b36df6d270c8b5d7e3ece0282664076c7b456b052fe0fd1
Sep 14 08:17:35.194020 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:35.193977    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266" event={"ID":"a73bb590-a5a9-4094-8e4e-0d235f10752e","Type":"ContainerStarted","Data":"ae95fe388b11afffb3c5c7ee3279f6ac80208d078c517e012b80cdb0d53f4d4b"}
Sep 14 08:17:35.194184 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:35.194030    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266" event={"ID":"a73bb590-a5a9-4094-8e4e-0d235f10752e","Type":"ContainerStarted","Data":"63d78a8ece16785b2b36df6d270c8b5d7e3ece0282664076c7b456b052fe0fd1"}
Sep 14 08:17:35.210267 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:35.210216    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266" podStartSLOduration=1.210202036 podStartE2EDuration="1.210202036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:17:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:17:35.208456775 +0000 UTC m=+1757.664906782" watchObservedRunningTime="2026-09-14 08:17:35.210202036 +0000 UTC m=+1757.666652070"
Sep 14 08:17:38.404257 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:38.404212    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:17:44.810301 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:44.810261    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:44.810727 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:44.810321    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:44.822751 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:44.822725    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:45.255830 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:45.255801    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:17:52.551992 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:52.551959    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:17:59.500832 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:17:59.500795    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:00.397221 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:00.397183    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:18:07.199176 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:07.199140    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:07.513382 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:07.513288    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:07.885384 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:07.885300    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:10.737024 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:10.736923    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:10.837536 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:10.837482    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:10.937652 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:10.937610    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:11.730164 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:11.730130    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:12.711425 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:12.711388    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:13.434488 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:13.434456    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:14.302264 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.302233    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"]
Sep 14 08:18:14.306306 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.306280    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"
Sep 14 08:18:14.308618 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.308591    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-925f2a-serving-cert\""
Sep 14 08:18:14.317038 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.317012    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"]
Sep 14 08:18:14.337654 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.337616    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:14.394377 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.394340    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:14.447551 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.447480    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sx22\" (UniqueName: \"kubernetes.io/projected/033c9734-a897-4019-b7b7-cb8322580425-kube-api-access-8sx22\") pod \"maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57\" (UID: \"033c9734-a897-4019-b7b7-cb8322580425\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"
Sep 14 08:18:14.447730 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.447644    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/033c9734-a897-4019-b7b7-cb8322580425-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57\" (UID: \"033c9734-a897-4019-b7b7-cb8322580425\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"
Sep 14 08:18:14.447730 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.447672    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/033c9734-a897-4019-b7b7-cb8322580425-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57\" (UID: \"033c9734-a897-4019-b7b7-cb8322580425\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"
Sep 14 08:18:14.447730 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.447705    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/033c9734-a897-4019-b7b7-cb8322580425-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57\" (UID: \"033c9734-a897-4019-b7b7-cb8322580425\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"
Sep 14 08:18:14.495995 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.495955    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:14.549058 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.549019    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/033c9734-a897-4019-b7b7-cb8322580425-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57\" (UID: \"033c9734-a897-4019-b7b7-cb8322580425\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"
Sep 14 08:18:14.549058 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.549065    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/033c9734-a897-4019-b7b7-cb8322580425-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57\" (UID: \"033c9734-a897-4019-b7b7-cb8322580425\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"
Sep 14 08:18:14.549308 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.549242    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/033c9734-a897-4019-b7b7-cb8322580425-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57\" (UID: \"033c9734-a897-4019-b7b7-cb8322580425\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"
Sep 14 08:18:14.549308 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.549293    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sx22\" (UniqueName: \"kubernetes.io/projected/033c9734-a897-4019-b7b7-cb8322580425-kube-api-access-8sx22\") pod \"maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57\" (UID: \"033c9734-a897-4019-b7b7-cb8322580425\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"
Sep 14 08:18:14.549900 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.549882    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/033c9734-a897-4019-b7b7-cb8322580425-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57\" (UID: \"033c9734-a897-4019-b7b7-cb8322580425\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"
Sep 14 08:18:14.551543 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.551496    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/033c9734-a897-4019-b7b7-cb8322580425-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57\" (UID: \"033c9734-a897-4019-b7b7-cb8322580425\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"
Sep 14 08:18:14.551646 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.551583    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/033c9734-a897-4019-b7b7-cb8322580425-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57\" (UID: \"033c9734-a897-4019-b7b7-cb8322580425\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"
Sep 14 08:18:14.556733 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.556676    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sx22\" (UniqueName: \"kubernetes.io/projected/033c9734-a897-4019-b7b7-cb8322580425-kube-api-access-8sx22\") pod \"maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57\" (UID: \"033c9734-a897-4019-b7b7-cb8322580425\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"
Sep 14 08:18:14.619596 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.619557    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"
Sep 14 08:18:14.808038 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.807948    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:14.890631 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.890599    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"]
Sep 14 08:18:14.892809 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:18:14.892774    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod033c9734_a897_4019_b7b7_cb8322580425.slice/crio-44cd35d9f285e66471686df3ff1664aabf45866f74627aeb57c8359736c016fd WatchSource:0}: Error finding container 44cd35d9f285e66471686df3ff1664aabf45866f74627aeb57c8359736c016fd: Status 404 returned error can't find the container with id 44cd35d9f285e66471686df3ff1664aabf45866f74627aeb57c8359736c016fd
Sep 14 08:18:14.892955 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.892864    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:14.894261 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:14.894242    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 08:18:15.222805 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:15.222764    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:15.397359 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:15.397316    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57" event={"ID":"033c9734-a897-4019-b7b7-cb8322580425","Type":"ContainerStarted","Data":"6ddd62b699f5112fc2b6c4faf77d33f5bfab03901db7332969a22ce3de510ac4"}
Sep 14 08:18:15.397810 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:15.397389    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"
Sep 14 08:18:15.397810 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:15.397408    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57" event={"ID":"033c9734-a897-4019-b7b7-cb8322580425","Type":"ContainerStarted","Data":"44cd35d9f285e66471686df3ff1664aabf45866f74627aeb57c8359736c016fd"}
Sep 14 08:18:15.415197 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:15.415140    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57" podStartSLOduration=0.99997459 podStartE2EDuration="1.41512445s" totalImagesPullingTime="415.14986ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:18:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:18:15.413424156 +0000 UTC m=+1797.869874150" watchObservedRunningTime="2026-09-14 08:18:15.41512445 +0000 UTC m=+1797.871574444"
Sep 14 08:18:16.034861 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:16.034828    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:19.897356 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:19.897312    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:18:21.408523 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:21.408470    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"
Sep 14 08:18:36.426220 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:36.426180    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:37.113726 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:37.113688    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:37.910113 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:37.910073    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:38.601654 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:38.601613    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:39.289087 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:39.289050    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:40.025542 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:40.025486    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:40.945384 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:40.945352    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:42.535250 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:42.535213    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:43.321416 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:43.321369    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:44.018769 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:44.018731    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:44.707166 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:44.707113    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:45.414350 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:45.414315    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:46.227633 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:46.227589    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:49.794125 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:49.794078    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:52.304088 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:52.304037    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:52.899646 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:52.899605    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"]
Sep 14 08:18:52.899870 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:52.899844    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57" podUID="033c9734-a897-4019-b7b7-cb8322580425" containerName="maas-api" containerID="cri-o://6ddd62b699f5112fc2b6c4faf77d33f5bfab03901db7332969a22ce3de510ac4" gracePeriod=30
Sep 14 08:18:52.930732 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:52.930693    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:53.002195 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.002154    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:53.061286 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.061250    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:53.144292 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.144267    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"
Sep 14 08:18:53.162927 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.162844    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:53.219926 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.219882    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:53.227234 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.227203    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/033c9734-a897-4019-b7b7-cb8322580425-maas-api-tls\") pod \"033c9734-a897-4019-b7b7-cb8322580425\" (UID: \"033c9734-a897-4019-b7b7-cb8322580425\") "
Sep 14 08:18:53.227373 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.227251    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/033c9734-a897-4019-b7b7-cb8322580425-openshift-service-ca\") pod \"033c9734-a897-4019-b7b7-cb8322580425\" (UID: \"033c9734-a897-4019-b7b7-cb8322580425\") "
Sep 14 08:18:53.227373 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.227276    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8sx22\" (UniqueName: \"kubernetes.io/projected/033c9734-a897-4019-b7b7-cb8322580425-kube-api-access-8sx22\") pod \"033c9734-a897-4019-b7b7-cb8322580425\" (UID: \"033c9734-a897-4019-b7b7-cb8322580425\") "
Sep 14 08:18:53.227373 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.227323    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/033c9734-a897-4019-b7b7-cb8322580425-metrics-certs\") pod \"033c9734-a897-4019-b7b7-cb8322580425\" (UID: \"033c9734-a897-4019-b7b7-cb8322580425\") "
Sep 14 08:18:53.227700 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.227659    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/033c9734-a897-4019-b7b7-cb8322580425-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "033c9734-a897-4019-b7b7-cb8322580425" (UID: "033c9734-a897-4019-b7b7-cb8322580425"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:18:53.229510 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.229464    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/033c9734-a897-4019-b7b7-cb8322580425-kube-api-access-8sx22" (OuterVolumeSpecName: "kube-api-access-8sx22") pod "033c9734-a897-4019-b7b7-cb8322580425" (UID: "033c9734-a897-4019-b7b7-cb8322580425"). InnerVolumeSpecName "kube-api-access-8sx22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:18:53.229614 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.229547    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/033c9734-a897-4019-b7b7-cb8322580425-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "033c9734-a897-4019-b7b7-cb8322580425" (UID: "033c9734-a897-4019-b7b7-cb8322580425"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:18:53.229614 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.229570    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/033c9734-a897-4019-b7b7-cb8322580425-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "033c9734-a897-4019-b7b7-cb8322580425" (UID: "033c9734-a897-4019-b7b7-cb8322580425"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:18:53.328943 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.328904    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/033c9734-a897-4019-b7b7-cb8322580425-metrics-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:18:53.328943 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.328936    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/033c9734-a897-4019-b7b7-cb8322580425-maas-api-tls\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:18:53.328943 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.328946    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/033c9734-a897-4019-b7b7-cb8322580425-openshift-service-ca\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:18:53.329381 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.328955    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8sx22\" (UniqueName: \"kubernetes.io/projected/033c9734-a897-4019-b7b7-cb8322580425-kube-api-access-8sx22\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:18:53.543361 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.543324    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:53.600527 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.600478    2562 generic.go:356] "Generic (PLEG): container finished" podID="033c9734-a897-4019-b7b7-cb8322580425" containerID="6ddd62b699f5112fc2b6c4faf77d33f5bfab03901db7332969a22ce3de510ac4" exitCode=0
Sep 14 08:18:53.600700 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.600543    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"
Sep 14 08:18:53.600700 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.600569    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57" event={"ID":"033c9734-a897-4019-b7b7-cb8322580425","Type":"ContainerDied","Data":"6ddd62b699f5112fc2b6c4faf77d33f5bfab03901db7332969a22ce3de510ac4"}
Sep 14 08:18:53.600700 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.600624    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57" event={"ID":"033c9734-a897-4019-b7b7-cb8322580425","Type":"ContainerDied","Data":"44cd35d9f285e66471686df3ff1664aabf45866f74627aeb57c8359736c016fd"}
Sep 14 08:18:53.600700 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.600647    2562 scope.go:117] "RemoveContainer" containerID="6ddd62b699f5112fc2b6c4faf77d33f5bfab03901db7332969a22ce3de510ac4"
Sep 14 08:18:53.610197 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.610180    2562 scope.go:117] "RemoveContainer" containerID="6ddd62b699f5112fc2b6c4faf77d33f5bfab03901db7332969a22ce3de510ac4"
Sep 14 08:18:53.610471 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:18:53.610455    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ddd62b699f5112fc2b6c4faf77d33f5bfab03901db7332969a22ce3de510ac4\": container with ID starting with 6ddd62b699f5112fc2b6c4faf77d33f5bfab03901db7332969a22ce3de510ac4 not found: ID does not exist" containerID="6ddd62b699f5112fc2b6c4faf77d33f5bfab03901db7332969a22ce3de510ac4"
Sep 14 08:18:53.610540 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.610480    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ddd62b699f5112fc2b6c4faf77d33f5bfab03901db7332969a22ce3de510ac4"} err="failed to get container status \"6ddd62b699f5112fc2b6c4faf77d33f5bfab03901db7332969a22ce3de510ac4\": rpc error: code = NotFound desc = could not find container \"6ddd62b699f5112fc2b6c4faf77d33f5bfab03901db7332969a22ce3de510ac4\": container with ID starting with 6ddd62b699f5112fc2b6c4faf77d33f5bfab03901db7332969a22ce3de510ac4 not found: ID does not exist"
Sep 14 08:18:53.620538 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.620473    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"]
Sep 14 08:18:53.623140 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.623118    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:53.626238 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:53.626216    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-925f2a-6cfcf8fd9d-qnv57"]
Sep 14 08:18:54.112923 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:54.112885    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="033c9734-a897-4019-b7b7-cb8322580425" path="/var/lib/kubelet/pods/033c9734-a897-4019-b7b7-cb8322580425/volumes"
Sep 14 08:18:54.327611 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:54.327579    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:55.302701 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:55.302663    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:56.145764 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:56.145681    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:56.851447 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:56.851409    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:57.698897 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:57.698858    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:18:59.515318 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:59.515274    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:59.616142 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:59.616099    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:18:59.773091 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:18:59.773005    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:03.693183 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:03.693147    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:04.436043 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:04.436009    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:05.420671 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:05.420631    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:06.135032 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:06.134996    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:06.944116 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:06.944078    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:07.599304 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:07.599257    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:16.510658 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:16.510620    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:19:18.295951 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:18.295922    2562 scope.go:117] "RemoveContainer" containerID="6e027226724a85e2a93a2cae971d8e910b7481616fde32e0e9bf61e96523ca5b"
Sep 14 08:19:18.925310 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:18.925266    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"]
Sep 14 08:19:18.926032 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:18.925993    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz" podUID="3cf481b3-bae6-4b5b-93aa-8a90f578cef8" containerName="maas-api" containerID="cri-o://e04a58fddd2818e233526e775cf39db1457da46cc74188f28a6ef9ddc74a7d84" gracePeriod=30
Sep 14 08:19:18.953403 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:18.953368    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:19.176619 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:19.176558    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"
Sep 14 08:19:19.278248 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:19.278213    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-openshift-service-ca\") pod \"3cf481b3-bae6-4b5b-93aa-8a90f578cef8\" (UID: \"3cf481b3-bae6-4b5b-93aa-8a90f578cef8\") "
Sep 14 08:19:19.278445 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:19.278258    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-maas-api-tls\") pod \"3cf481b3-bae6-4b5b-93aa-8a90f578cef8\" (UID: \"3cf481b3-bae6-4b5b-93aa-8a90f578cef8\") "
Sep 14 08:19:19.278445 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:19.278329    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-metrics-certs\") pod \"3cf481b3-bae6-4b5b-93aa-8a90f578cef8\" (UID: \"3cf481b3-bae6-4b5b-93aa-8a90f578cef8\") "
Sep 14 08:19:19.278609 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:19.278567    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67glr\" (UniqueName: \"kubernetes.io/projected/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-kube-api-access-67glr\") pod \"3cf481b3-bae6-4b5b-93aa-8a90f578cef8\" (UID: \"3cf481b3-bae6-4b5b-93aa-8a90f578cef8\") "
Sep 14 08:19:19.278715 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:19.278690    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3cf481b3-bae6-4b5b-93aa-8a90f578cef8" (UID: "3cf481b3-bae6-4b5b-93aa-8a90f578cef8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:19:19.278954 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:19.278929    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-openshift-service-ca\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:19:19.280486 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:19.280461    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3cf481b3-bae6-4b5b-93aa-8a90f578cef8" (UID: "3cf481b3-bae6-4b5b-93aa-8a90f578cef8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:19:19.280608 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:19.280491    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3cf481b3-bae6-4b5b-93aa-8a90f578cef8" (UID: "3cf481b3-bae6-4b5b-93aa-8a90f578cef8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:19:19.280608 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:19.280566    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-kube-api-access-67glr" (OuterVolumeSpecName: "kube-api-access-67glr") pod "3cf481b3-bae6-4b5b-93aa-8a90f578cef8" (UID: "3cf481b3-bae6-4b5b-93aa-8a90f578cef8"). InnerVolumeSpecName "kube-api-access-67glr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:19:19.380312 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:19.380274    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-metrics-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:19:19.380312 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:19.380310    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67glr\" (UniqueName: \"kubernetes.io/projected/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-kube-api-access-67glr\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:19:19.380312 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:19.380321    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cf481b3-bae6-4b5b-93aa-8a90f578cef8-maas-api-tls\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:19:19.738001 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:19.737969    2562 generic.go:356] "Generic (PLEG): container finished" podID="3cf481b3-bae6-4b5b-93aa-8a90f578cef8" containerID="e04a58fddd2818e233526e775cf39db1457da46cc74188f28a6ef9ddc74a7d84" exitCode=0
Sep 14 08:19:19.738187 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:19.738008    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"
Sep 14 08:19:19.738187 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:19.738052    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz" event={"ID":"3cf481b3-bae6-4b5b-93aa-8a90f578cef8","Type":"ContainerDied","Data":"e04a58fddd2818e233526e775cf39db1457da46cc74188f28a6ef9ddc74a7d84"}
Sep 14 08:19:19.738187 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:19.738127    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz" event={"ID":"3cf481b3-bae6-4b5b-93aa-8a90f578cef8","Type":"ContainerDied","Data":"86c8c6ffd9f0aa739bfd3c8a6a0abaeaa3a95288587a2219bdbe9a357a7f6f4b"}
Sep 14 08:19:19.738187 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:19.738150    2562 scope.go:117] "RemoveContainer" containerID="e04a58fddd2818e233526e775cf39db1457da46cc74188f28a6ef9ddc74a7d84"
Sep 14 08:19:19.747293 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:19.747268    2562 scope.go:117] "RemoveContainer" containerID="e04a58fddd2818e233526e775cf39db1457da46cc74188f28a6ef9ddc74a7d84"
Sep 14 08:19:19.747575 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:19:19.747554    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e04a58fddd2818e233526e775cf39db1457da46cc74188f28a6ef9ddc74a7d84\": container with ID starting with e04a58fddd2818e233526e775cf39db1457da46cc74188f28a6ef9ddc74a7d84 not found: ID does not exist" containerID="e04a58fddd2818e233526e775cf39db1457da46cc74188f28a6ef9ddc74a7d84"
Sep 14 08:19:19.747659 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:19.747583    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e04a58fddd2818e233526e775cf39db1457da46cc74188f28a6ef9ddc74a7d84"} err="failed to get container status \"e04a58fddd2818e233526e775cf39db1457da46cc74188f28a6ef9ddc74a7d84\": rpc error: code = NotFound desc = could not find container \"e04a58fddd2818e233526e775cf39db1457da46cc74188f28a6ef9ddc74a7d84\": container with ID starting with e04a58fddd2818e233526e775cf39db1457da46cc74188f28a6ef9ddc74a7d84 not found: ID does not exist"
Sep 14 08:19:19.758873 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:19.758849    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"]
Sep 14 08:19:19.762141 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:19.762120    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7f1623-769d5d5c7-68dqz"]
Sep 14 08:19:20.112838 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:20.112761    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cf481b3-bae6-4b5b-93aa-8a90f578cef8" path="/var/lib/kubelet/pods/3cf481b3-bae6-4b5b-93aa-8a90f578cef8/volumes"
Sep 14 08:19:20.329990 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:20.329946    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:20.442732 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:20.442705    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:21.256137 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:21.256106    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:22.149002 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:22.148969    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:24.495237 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:24.495200    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"]
Sep 14 08:19:24.590771 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:24.590724    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:19:30.103114 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:30.103079    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:19:39.514950 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.514890    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266" podUID="a73bb590-a5a9-4094-8e4e-0d235f10752e" containerName="main" containerID="cri-o://ae95fe388b11afffb3c5c7ee3279f6ac80208d078c517e012b80cdb0d53f4d4b" gracePeriod=45
Sep 14 08:19:39.774817 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.774752    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:19:39.835466 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.835421    2562 generic.go:356] "Generic (PLEG): container finished" podID="a73bb590-a5a9-4094-8e4e-0d235f10752e" containerID="ae95fe388b11afffb3c5c7ee3279f6ac80208d078c517e012b80cdb0d53f4d4b" exitCode=0
Sep 14 08:19:39.835661 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.835472    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"
Sep 14 08:19:39.835661 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.835493    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266" event={"ID":"a73bb590-a5a9-4094-8e4e-0d235f10752e","Type":"ContainerDied","Data":"ae95fe388b11afffb3c5c7ee3279f6ac80208d078c517e012b80cdb0d53f4d4b"}
Sep 14 08:19:39.835661 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.835559    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266" event={"ID":"a73bb590-a5a9-4094-8e4e-0d235f10752e","Type":"ContainerDied","Data":"63d78a8ece16785b2b36df6d270c8b5d7e3ece0282664076c7b456b052fe0fd1"}
Sep 14 08:19:39.835661 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.835579    2562 scope.go:117] "RemoveContainer" containerID="ae95fe388b11afffb3c5c7ee3279f6ac80208d078c517e012b80cdb0d53f4d4b"
Sep 14 08:19:39.844215 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.844196    2562 scope.go:117] "RemoveContainer" containerID="ae95fe388b11afffb3c5c7ee3279f6ac80208d078c517e012b80cdb0d53f4d4b"
Sep 14 08:19:39.844485 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:19:39.844465    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae95fe388b11afffb3c5c7ee3279f6ac80208d078c517e012b80cdb0d53f4d4b\": container with ID starting with ae95fe388b11afffb3c5c7ee3279f6ac80208d078c517e012b80cdb0d53f4d4b not found: ID does not exist" containerID="ae95fe388b11afffb3c5c7ee3279f6ac80208d078c517e012b80cdb0d53f4d4b"
Sep 14 08:19:39.844576 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.844515    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae95fe388b11afffb3c5c7ee3279f6ac80208d078c517e012b80cdb0d53f4d4b"} err="failed to get container status \"ae95fe388b11afffb3c5c7ee3279f6ac80208d078c517e012b80cdb0d53f4d4b\": rpc error: code = NotFound desc = could not find container \"ae95fe388b11afffb3c5c7ee3279f6ac80208d078c517e012b80cdb0d53f4d4b\": container with ID starting with ae95fe388b11afffb3c5c7ee3279f6ac80208d078c517e012b80cdb0d53f4d4b not found: ID does not exist"
Sep 14 08:19:39.886305 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.886271    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5t2d\" (UniqueName: \"kubernetes.io/projected/a73bb590-a5a9-4094-8e4e-0d235f10752e-kube-api-access-m5t2d\") pod \"a73bb590-a5a9-4094-8e4e-0d235f10752e\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") "
Sep 14 08:19:39.886524 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.886320    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-tmp-dir\") pod \"a73bb590-a5a9-4094-8e4e-0d235f10752e\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") "
Sep 14 08:19:39.886524 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.886353    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-model-cache\") pod \"a73bb590-a5a9-4094-8e4e-0d235f10752e\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") "
Sep 14 08:19:39.886524 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.886394    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a73bb590-a5a9-4094-8e4e-0d235f10752e-tls-certs\") pod \"a73bb590-a5a9-4094-8e4e-0d235f10752e\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") "
Sep 14 08:19:39.886524 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.886461    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-dshm\") pod \"a73bb590-a5a9-4094-8e4e-0d235f10752e\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") "
Sep 14 08:19:39.886703 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.886603    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-home\") pod \"a73bb590-a5a9-4094-8e4e-0d235f10752e\" (UID: \"a73bb590-a5a9-4094-8e4e-0d235f10752e\") "
Sep 14 08:19:39.886703 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.886632    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a73bb590-a5a9-4094-8e4e-0d235f10752e" (UID: "a73bb590-a5a9-4094-8e4e-0d235f10752e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:19:39.886703 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.886646    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-model-cache" (OuterVolumeSpecName: "model-cache") pod "a73bb590-a5a9-4094-8e4e-0d235f10752e" (UID: "a73bb590-a5a9-4094-8e4e-0d235f10752e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:19:39.886937 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.886908    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-home" (OuterVolumeSpecName: "home") pod "a73bb590-a5a9-4094-8e4e-0d235f10752e" (UID: "a73bb590-a5a9-4094-8e4e-0d235f10752e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:19:39.887017 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.886966    2562 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-tmp-dir\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:19:39.887017 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.886982    2562 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-model-cache\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:19:39.888666 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.888633    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-dshm" (OuterVolumeSpecName: "dshm") pod "a73bb590-a5a9-4094-8e4e-0d235f10752e" (UID: "a73bb590-a5a9-4094-8e4e-0d235f10752e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:19:39.888767 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.888663    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a73bb590-a5a9-4094-8e4e-0d235f10752e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a73bb590-a5a9-4094-8e4e-0d235f10752e" (UID: "a73bb590-a5a9-4094-8e4e-0d235f10752e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:19:39.888767 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.888728    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a73bb590-a5a9-4094-8e4e-0d235f10752e-kube-api-access-m5t2d" (OuterVolumeSpecName: "kube-api-access-m5t2d") pod "a73bb590-a5a9-4094-8e4e-0d235f10752e" (UID: "a73bb590-a5a9-4094-8e4e-0d235f10752e"). InnerVolumeSpecName "kube-api-access-m5t2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:19:39.988412 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.988376    2562 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-dshm\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:19:39.988412 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.988404    2562 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a73bb590-a5a9-4094-8e4e-0d235f10752e-home\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:19:39.988412 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.988414    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5t2d\" (UniqueName: \"kubernetes.io/projected/a73bb590-a5a9-4094-8e4e-0d235f10752e-kube-api-access-m5t2d\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:19:39.988683 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:39.988424    2562 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a73bb590-a5a9-4094-8e4e-0d235f10752e-tls-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:19:40.151331 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:40.151296    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"]
Sep 14 08:19:40.155048 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:40.155018    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-7f1623/ipp-route-7f1623-kserve-668bcc7b95-tg266"]
Sep 14 08:19:42.112675 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:19:42.112644    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a73bb590-a5a9-4094-8e4e-0d235f10752e" path="/var/lib/kubelet/pods/a73bb590-a5a9-4094-8e4e-0d235f10752e/volumes"
Sep 14 08:20:00.717269 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:00.717231    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:01.479560 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:01.479495    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:02.959877 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:02.959841    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:07.122174 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:07.122137    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:12.735538 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:12.735479    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:12.836127 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:12.836091    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:13.011351 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:13.011265    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:13.541279 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:13.541240    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:14.416329 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:14.416297    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:15.186442 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:15.186409    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:15.917785 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:15.917747    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:16.742934 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:16.742890    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:18.319400 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:18.319364    2562 scope.go:117] "RemoveContainer" containerID="4140fe56933b97dda88d275c062f289c5d5a1527bd8d5eb438ab6861dc7e8c72"
Sep 14 08:20:23.852479 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:23.852447    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"]
Sep 14 08:20:23.853089 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:23.852671    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf" podUID="c2387d0f-4dba-412b-99f3-60591216d4e9" containerName="maas-api" containerID="cri-o://3f69d25425b3b96b466e3eb9a4e633b469cc49a629b8643544197b05c7713b63" gracePeriod=30
Sep 14 08:20:23.880474 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:23.880437    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:24.052790 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:24.052756    2562 generic.go:356] "Generic (PLEG): container finished" podID="c2387d0f-4dba-412b-99f3-60591216d4e9" containerID="3f69d25425b3b96b466e3eb9a4e633b469cc49a629b8643544197b05c7713b63" exitCode=0
Sep 14 08:20:24.052987 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:24.052824    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf" event={"ID":"c2387d0f-4dba-412b-99f3-60591216d4e9","Type":"ContainerDied","Data":"3f69d25425b3b96b466e3eb9a4e633b469cc49a629b8643544197b05c7713b63"}
Sep 14 08:20:24.093810 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:24.093779    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"
Sep 14 08:20:24.100356 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:24.100327    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:24.184358 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:24.184329    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:24.200329 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:24.200297    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2387d0f-4dba-412b-99f3-60591216d4e9-metrics-certs\") pod \"c2387d0f-4dba-412b-99f3-60591216d4e9\" (UID: \"c2387d0f-4dba-412b-99f3-60591216d4e9\") "
Sep 14 08:20:24.200487 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:24.200367    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2387d0f-4dba-412b-99f3-60591216d4e9-openshift-service-ca\") pod \"c2387d0f-4dba-412b-99f3-60591216d4e9\" (UID: \"c2387d0f-4dba-412b-99f3-60591216d4e9\") "
Sep 14 08:20:24.200487 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:24.200414    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2387d0f-4dba-412b-99f3-60591216d4e9-maas-api-tls\") pod \"c2387d0f-4dba-412b-99f3-60591216d4e9\" (UID: \"c2387d0f-4dba-412b-99f3-60591216d4e9\") "
Sep 14 08:20:24.200487 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:24.200452    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwlrh\" (UniqueName: \"kubernetes.io/projected/c2387d0f-4dba-412b-99f3-60591216d4e9-kube-api-access-kwlrh\") pod \"c2387d0f-4dba-412b-99f3-60591216d4e9\" (UID: \"c2387d0f-4dba-412b-99f3-60591216d4e9\") "
Sep 14 08:20:24.200810 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:24.200779    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2387d0f-4dba-412b-99f3-60591216d4e9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c2387d0f-4dba-412b-99f3-60591216d4e9" (UID: "c2387d0f-4dba-412b-99f3-60591216d4e9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:20:24.202679 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:24.202650    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2387d0f-4dba-412b-99f3-60591216d4e9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c2387d0f-4dba-412b-99f3-60591216d4e9" (UID: "c2387d0f-4dba-412b-99f3-60591216d4e9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:20:24.202679 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:24.202664    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2387d0f-4dba-412b-99f3-60591216d4e9-kube-api-access-kwlrh" (OuterVolumeSpecName: "kube-api-access-kwlrh") pod "c2387d0f-4dba-412b-99f3-60591216d4e9" (UID: "c2387d0f-4dba-412b-99f3-60591216d4e9"). InnerVolumeSpecName "kube-api-access-kwlrh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:20:24.202679 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:24.202677    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2387d0f-4dba-412b-99f3-60591216d4e9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c2387d0f-4dba-412b-99f3-60591216d4e9" (UID: "c2387d0f-4dba-412b-99f3-60591216d4e9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:20:24.301487 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:24.301443    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2387d0f-4dba-412b-99f3-60591216d4e9-maas-api-tls\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:20:24.301487 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:24.301488    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwlrh\" (UniqueName: \"kubernetes.io/projected/c2387d0f-4dba-412b-99f3-60591216d4e9-kube-api-access-kwlrh\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:20:24.301743 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:24.301551    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2387d0f-4dba-412b-99f3-60591216d4e9-metrics-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:20:24.301743 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:24.301638    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2387d0f-4dba-412b-99f3-60591216d4e9-openshift-service-ca\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:20:24.502338 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:24.502253    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:24.601274 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:24.601238    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:25.059416 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:25.059381    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"
Sep 14 08:20:25.059416 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:25.059400    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf" event={"ID":"c2387d0f-4dba-412b-99f3-60591216d4e9","Type":"ContainerDied","Data":"f7892030fe6cb720f1a8828f3c611eb988623dcb70a94285a94a73dcfd848d44"}
Sep 14 08:20:25.059968 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:25.059465    2562 scope.go:117] "RemoveContainer" containerID="3f69d25425b3b96b466e3eb9a4e633b469cc49a629b8643544197b05c7713b63"
Sep 14 08:20:25.081781 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:25.081751    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"]
Sep 14 08:20:25.087080 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:25.087053    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-56708c-5498d7fd96-9tzwf"]
Sep 14 08:20:25.343211 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:25.343133    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:26.040566 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:26.040522    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:26.114243 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:26.114206    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2387d0f-4dba-412b-99f3-60591216d4e9" path="/var/lib/kubelet/pods/c2387d0f-4dba-412b-99f3-60591216d4e9/volumes"
Sep 14 08:20:27.094696 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:27.094659    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:20:30.267276 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:30.267236    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:30.332822 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:30.332782    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:30.543264 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:30.543182    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:30.886747 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:30.886662    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-7746e4-psc59"]
Sep 14 08:20:30.887052 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:30.887040    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2387d0f-4dba-412b-99f3-60591216d4e9" containerName="maas-api"
Sep 14 08:20:30.887098 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:30.887054    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2387d0f-4dba-412b-99f3-60591216d4e9" containerName="maas-api"
Sep 14 08:20:30.887098 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:30.887062    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a73bb590-a5a9-4094-8e4e-0d235f10752e" containerName="main"
Sep 14 08:20:30.887098 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:30.887067    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a73bb590-a5a9-4094-8e4e-0d235f10752e" containerName="main"
Sep 14 08:20:30.887098 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:30.887079    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="033c9734-a897-4019-b7b7-cb8322580425" containerName="maas-api"
Sep 14 08:20:30.887098 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:30.887087    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="033c9734-a897-4019-b7b7-cb8322580425" containerName="maas-api"
Sep 14 08:20:30.887098 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:30.887094    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf481b3-bae6-4b5b-93aa-8a90f578cef8" containerName="maas-api"
Sep 14 08:20:30.887288 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:30.887099    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf481b3-bae6-4b5b-93aa-8a90f578cef8" containerName="maas-api"
Sep 14 08:20:30.887288 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:30.887154    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="033c9734-a897-4019-b7b7-cb8322580425" containerName="maas-api"
Sep 14 08:20:30.887288 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:30.887163    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2387d0f-4dba-412b-99f3-60591216d4e9" containerName="maas-api"
Sep 14 08:20:30.887288 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:30.887172    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a73bb590-a5a9-4094-8e4e-0d235f10752e" containerName="main"
Sep 14 08:20:30.887288 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:30.887177    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cf481b3-bae6-4b5b-93aa-8a90f578cef8" containerName="maas-api"
Sep 14 08:20:30.891552 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:30.891531    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-7746e4-psc59"
Sep 14 08:20:30.893608 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:30.893580    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wkgn2\""
Sep 14 08:20:30.893738 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:30.893612    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:20:30.897477 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:30.897449    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-7746e4-psc59"]
Sep 14 08:20:30.962045 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:30.961998    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc69bf26-aab4-480b-b0a5-ad4beaa7606e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-7746e4-psc59\" (UID: \"dc69bf26-aab4-480b-b0a5-ad4beaa7606e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-7746e4-psc59"
Sep 14 08:20:31.062909 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:31.062869    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc69bf26-aab4-480b-b0a5-ad4beaa7606e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-7746e4-psc59\" (UID: \"dc69bf26-aab4-480b-b0a5-ad4beaa7606e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-7746e4-psc59"
Sep 14 08:20:31.063542 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:31.063493    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc69bf26-aab4-480b-b0a5-ad4beaa7606e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-7746e4-psc59\" (UID: \"dc69bf26-aab4-480b-b0a5-ad4beaa7606e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-7746e4-psc59"
Sep 14 08:20:31.202814 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:31.202774    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-7746e4-psc59"
Sep 14 08:20:31.478786 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:31.478754    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-7746e4-psc59"]
Sep 14 08:20:31.481347 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:20:31.481317    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc69bf26_aab4_480b_b0a5_ad4beaa7606e.slice/crio-ddc63e8cab5071f25fb09e313cb5ae9e1b266beb5038d4bab03a50b4f30e938e WatchSource:0}: Error finding container ddc63e8cab5071f25fb09e313cb5ae9e1b266beb5038d4bab03a50b4f30e938e: Status 404 returned error can't find the container with id ddc63e8cab5071f25fb09e313cb5ae9e1b266beb5038d4bab03a50b4f30e938e
Sep 14 08:20:31.957529 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:31.957479    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:32.097108 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:32.097014    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-7746e4-psc59" event={"ID":"dc69bf26-aab4-480b-b0a5-ad4beaa7606e","Type":"ContainerStarted","Data":"e1497e3bff09e023d9c7ecb9577d736235e9f0a1af5c59f850a6cac8e0067000"}
Sep 14 08:20:32.097108 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:32.097071    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-7746e4-psc59" event={"ID":"dc69bf26-aab4-480b-b0a5-ad4beaa7606e","Type":"ContainerStarted","Data":"ddc63e8cab5071f25fb09e313cb5ae9e1b266beb5038d4bab03a50b4f30e938e"}
Sep 14 08:20:32.118200 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:32.118145    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-7746e4-psc59" podStartSLOduration=2.118128672 podStartE2EDuration="2.118128672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:20:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:20:32.117055631 +0000 UTC m=+1934.573505636" watchObservedRunningTime="2026-09-14 08:20:32.118128672 +0000 UTC m=+1934.574578658"
Sep 14 08:20:32.727721 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:32.727683    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:33.103701 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:33.103605    2562 generic.go:356] "Generic (PLEG): container finished" podID="dc69bf26-aab4-480b-b0a5-ad4beaa7606e" containerID="e1497e3bff09e023d9c7ecb9577d736235e9f0a1af5c59f850a6cac8e0067000" exitCode=0
Sep 14 08:20:33.103701 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:33.103643    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-7746e4-psc59" event={"ID":"dc69bf26-aab4-480b-b0a5-ad4beaa7606e","Type":"ContainerDied","Data":"e1497e3bff09e023d9c7ecb9577d736235e9f0a1af5c59f850a6cac8e0067000"}
Sep 14 08:20:33.623724 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:33.623687    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:34.238646 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:34.238614    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-7746e4-psc59"
Sep 14 08:20:34.294218 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:34.294187    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc69bf26-aab4-480b-b0a5-ad4beaa7606e-maas-api-service-ca\") pod \"dc69bf26-aab4-480b-b0a5-ad4beaa7606e\" (UID: \"dc69bf26-aab4-480b-b0a5-ad4beaa7606e\") "
Sep 14 08:20:34.294590 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:34.294566    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc69bf26-aab4-480b-b0a5-ad4beaa7606e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dc69bf26-aab4-480b-b0a5-ad4beaa7606e" (UID: "dc69bf26-aab4-480b-b0a5-ad4beaa7606e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:20:34.395621 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:34.395532    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc69bf26-aab4-480b-b0a5-ad4beaa7606e-maas-api-service-ca\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:20:35.114465 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:35.114434    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-7746e4-psc59"
Sep 14 08:20:35.114666 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:35.114476    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-7746e4-psc59" event={"ID":"dc69bf26-aab4-480b-b0a5-ad4beaa7606e","Type":"ContainerDied","Data":"ddc63e8cab5071f25fb09e313cb5ae9e1b266beb5038d4bab03a50b4f30e938e"}
Sep 14 08:20:35.114666 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:35.114543    2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ddc63e8cab5071f25fb09e313cb5ae9e1b266beb5038d4bab03a50b4f30e938e"
Sep 14 08:20:41.439632 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:41.439601    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:41.446076 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:41.446049    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7746e4-29822895-dskdr"]
Sep 14 08:20:41.450912 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:41.450888    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-7746e4-29822895-dskdr"]
Sep 14 08:20:41.509241 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:41.509206    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:41.608985 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:41.608954    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:41.646806 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:41.646763    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-7746e4-psc59"]
Sep 14 08:20:41.649723 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:41.649696    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-7746e4-psc59"]
Sep 14 08:20:41.665935 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:41.665908    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:42.013629 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:42.013583    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:42.113414 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:42.113379    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67a60a44-f359-49c3-b88a-18a733262075" path="/var/lib/kubelet/pods/67a60a44-f359-49c3-b88a-18a733262075/volumes"
Sep 14 08:20:42.113754 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:42.113740    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc69bf26-aab4-480b-b0a5-ad4beaa7606e" path="/var/lib/kubelet/pods/dc69bf26-aab4-480b-b0a5-ad4beaa7606e/volumes"
Sep 14 08:20:42.115953 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:42.115922    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:42.890109 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:42.890064    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:44.401830 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:44.401796    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:20:48.180799 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:48.180762    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:48.281710 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:48.281672    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:48.432194 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:48.432102    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"]
Sep 14 08:20:48.432365 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:48.432323    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn" podUID="10943269-128b-4c50-82d8-e70c44c3a70f" containerName="istio-proxy" containerID="cri-o://a9efb7002bb14365936ada3400b796af886a107ad80cd1c391d590243203eaca" gracePeriod=30
Sep 14 08:20:48.453343 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:48.453307    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:49.533418 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:49.533384    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:49.753904 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:49.753864    2562 patch_prober.go:28] interesting pod/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 08:20:49.754065 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:49.753924    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn" podUID="10943269-128b-4c50-82d8-e70c44c3a70f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 08:20:50.310592 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:50.310555    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:53.686772 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.686743    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:20:53.788075 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.787977    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-istio-envoy\") pod \"10943269-128b-4c50-82d8-e70c44c3a70f\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") "
Sep 14 08:20:53.788075 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.788023    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-credential-socket\") pod \"10943269-128b-4c50-82d8-e70c44c3a70f\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") "
Sep 14 08:20:53.788075 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.788065    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10943269-128b-4c50-82d8-e70c44c3a70f-istio-token\") pod \"10943269-128b-4c50-82d8-e70c44c3a70f\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") "
Sep 14 08:20:53.788327 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.788101    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4hrd\" (UniqueName: \"kubernetes.io/projected/10943269-128b-4c50-82d8-e70c44c3a70f-kube-api-access-g4hrd\") pod \"10943269-128b-4c50-82d8-e70c44c3a70f\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") "
Sep 14 08:20:53.788327 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.788124    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10943269-128b-4c50-82d8-e70c44c3a70f-istio-podinfo\") pod \"10943269-128b-4c50-82d8-e70c44c3a70f\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") "
Sep 14 08:20:53.788327 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.788177    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10943269-128b-4c50-82d8-e70c44c3a70f-istiod-ca-cert\") pod \"10943269-128b-4c50-82d8-e70c44c3a70f\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") "
Sep 14 08:20:53.788327 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.788235    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-workload-socket\") pod \"10943269-128b-4c50-82d8-e70c44c3a70f\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") "
Sep 14 08:20:53.788574 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.788405    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "10943269-128b-4c50-82d8-e70c44c3a70f" (UID: "10943269-128b-4c50-82d8-e70c44c3a70f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:20:53.788574 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.788468    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-workload-certs\") pod \"10943269-128b-4c50-82d8-e70c44c3a70f\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") "
Sep 14 08:20:53.788684 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.788592    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "10943269-128b-4c50-82d8-e70c44c3a70f" (UID: "10943269-128b-4c50-82d8-e70c44c3a70f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:20:53.788684 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.788653    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-istio-data\") pod \"10943269-128b-4c50-82d8-e70c44c3a70f\" (UID: \"10943269-128b-4c50-82d8-e70c44c3a70f\") "
Sep 14 08:20:53.788780 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.788727    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "10943269-128b-4c50-82d8-e70c44c3a70f" (UID: "10943269-128b-4c50-82d8-e70c44c3a70f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:20:53.788903 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.788880    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-istio-data" (OuterVolumeSpecName: "istio-data") pod "10943269-128b-4c50-82d8-e70c44c3a70f" (UID: "10943269-128b-4c50-82d8-e70c44c3a70f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:20:53.789004 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.788944    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10943269-128b-4c50-82d8-e70c44c3a70f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "10943269-128b-4c50-82d8-e70c44c3a70f" (UID: "10943269-128b-4c50-82d8-e70c44c3a70f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:20:53.789279 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.789259    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-workload-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:20:53.789315 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.789287    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-workload-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:20:53.789315 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.789303    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-istio-data\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:20:53.789381 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.789319    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-credential-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:20:53.789381 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.789332    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10943269-128b-4c50-82d8-e70c44c3a70f-istiod-ca-cert\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:20:53.790288 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.790259    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10943269-128b-4c50-82d8-e70c44c3a70f-kube-api-access-g4hrd" (OuterVolumeSpecName: "kube-api-access-g4hrd") pod "10943269-128b-4c50-82d8-e70c44c3a70f" (UID: "10943269-128b-4c50-82d8-e70c44c3a70f"). InnerVolumeSpecName "kube-api-access-g4hrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:20:53.790384 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.790331    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10943269-128b-4c50-82d8-e70c44c3a70f-istio-token" (OuterVolumeSpecName: "istio-token") pod "10943269-128b-4c50-82d8-e70c44c3a70f" (UID: "10943269-128b-4c50-82d8-e70c44c3a70f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:20:53.790638 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.790617    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "10943269-128b-4c50-82d8-e70c44c3a70f" (UID: "10943269-128b-4c50-82d8-e70c44c3a70f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:20:53.790700 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.790644    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/10943269-128b-4c50-82d8-e70c44c3a70f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "10943269-128b-4c50-82d8-e70c44c3a70f" (UID: "10943269-128b-4c50-82d8-e70c44c3a70f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:20:53.890355 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.890315    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10943269-128b-4c50-82d8-e70c44c3a70f-istio-envoy\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:20:53.890355 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.890347    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10943269-128b-4c50-82d8-e70c44c3a70f-istio-token\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:20:53.890355 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.890357    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4hrd\" (UniqueName: \"kubernetes.io/projected/10943269-128b-4c50-82d8-e70c44c3a70f-kube-api-access-g4hrd\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:20:53.890355 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:53.890367    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10943269-128b-4c50-82d8-e70c44c3a70f-istio-podinfo\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:20:54.207828 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:54.207790    2562 generic.go:356] "Generic (PLEG): container finished" podID="10943269-128b-4c50-82d8-e70c44c3a70f" containerID="a9efb7002bb14365936ada3400b796af886a107ad80cd1c391d590243203eaca" exitCode=0
Sep 14 08:20:54.208005 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:54.207847    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"
Sep 14 08:20:54.208005 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:54.207881    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn" event={"ID":"10943269-128b-4c50-82d8-e70c44c3a70f","Type":"ContainerDied","Data":"a9efb7002bb14365936ada3400b796af886a107ad80cd1c391d590243203eaca"}
Sep 14 08:20:54.208005 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:54.207949    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn" event={"ID":"10943269-128b-4c50-82d8-e70c44c3a70f","Type":"ContainerDied","Data":"36a29923dbd521f2dbc52a20eeb397e25285b8b43499f0995cd0ff04f83969ab"}
Sep 14 08:20:54.208005 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:54.207978    2562 scope.go:117] "RemoveContainer" containerID="a9efb7002bb14365936ada3400b796af886a107ad80cd1c391d590243203eaca"
Sep 14 08:20:54.216905 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:54.216887    2562 scope.go:117] "RemoveContainer" containerID="5c8c2c56d5038986301d4735387235c528f2867d0ee1a5f0e5fab15ae8f097c7"
Sep 14 08:20:54.223580 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:54.223554    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"]
Sep 14 08:20:54.225615 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:54.225595    2562 scope.go:117] "RemoveContainer" containerID="a9efb7002bb14365936ada3400b796af886a107ad80cd1c391d590243203eaca"
Sep 14 08:20:54.226179 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:20:54.226108    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9efb7002bb14365936ada3400b796af886a107ad80cd1c391d590243203eaca\": container with ID starting with a9efb7002bb14365936ada3400b796af886a107ad80cd1c391d590243203eaca not found: ID does not exist" containerID="a9efb7002bb14365936ada3400b796af886a107ad80cd1c391d590243203eaca"
Sep 14 08:20:54.226294 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:54.226186    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9efb7002bb14365936ada3400b796af886a107ad80cd1c391d590243203eaca"} err="failed to get container status \"a9efb7002bb14365936ada3400b796af886a107ad80cd1c391d590243203eaca\": rpc error: code = NotFound desc = could not find container \"a9efb7002bb14365936ada3400b796af886a107ad80cd1c391d590243203eaca\": container with ID starting with a9efb7002bb14365936ada3400b796af886a107ad80cd1c391d590243203eaca not found: ID does not exist"
Sep 14 08:20:54.226294 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:54.226230    2562 scope.go:117] "RemoveContainer" containerID="5c8c2c56d5038986301d4735387235c528f2867d0ee1a5f0e5fab15ae8f097c7"
Sep 14 08:20:54.226705 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:20:54.226680    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c8c2c56d5038986301d4735387235c528f2867d0ee1a5f0e5fab15ae8f097c7\": container with ID starting with 5c8c2c56d5038986301d4735387235c528f2867d0ee1a5f0e5fab15ae8f097c7 not found: ID does not exist" containerID="5c8c2c56d5038986301d4735387235c528f2867d0ee1a5f0e5fab15ae8f097c7"
Sep 14 08:20:54.226781 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:54.226711    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c8c2c56d5038986301d4735387235c528f2867d0ee1a5f0e5fab15ae8f097c7"} err="failed to get container status \"5c8c2c56d5038986301d4735387235c528f2867d0ee1a5f0e5fab15ae8f097c7\": rpc error: code = NotFound desc = could not find container \"5c8c2c56d5038986301d4735387235c528f2867d0ee1a5f0e5fab15ae8f097c7\": container with ID starting with 5c8c2c56d5038986301d4735387235c528f2867d0ee1a5f0e5fab15ae8f097c7 not found: ID does not exist"
Sep 14 08:20:54.227717 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:54.227695    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-7746e4-openshift-default-55bc4c84db-kghvn"]
Sep 14 08:20:56.113352 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:56.113272    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10943269-128b-4c50-82d8-e70c44c3a70f" path="/var/lib/kubelet/pods/10943269-128b-4c50-82d8-e70c44c3a70f/volumes"
Sep 14 08:20:59.335297 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:59.335260    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:59.431749 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:59.431709    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:59.531252 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:59.531218    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:59.582317 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:59.582287    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:20:59.926067 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:20:59.926031    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:00.030267 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:00.030236    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:02.301747 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:02.301704    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:21:05.826468 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:05.826424    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:05.926661 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:05.926621    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:06.060226 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:06.060194    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"]
Sep 14 08:21:06.060455 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:06.060434    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf" podUID="c58ec321-5017-43e7-adbe-107cd0f3b7c4" containerName="istio-proxy" containerID="cri-o://2bd6ebe2b104f8dbaafd0eaf8b7d06c8fe5c2867a61c4b2b8b0dce89d9e01051" gracePeriod=30
Sep 14 08:21:06.080245 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:06.080145    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:10.597583 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:10.597476    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:21:11.292808 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.292776    2562 generic.go:356] "Generic (PLEG): container finished" podID="c58ec321-5017-43e7-adbe-107cd0f3b7c4" containerID="2bd6ebe2b104f8dbaafd0eaf8b7d06c8fe5c2867a61c4b2b8b0dce89d9e01051" exitCode=0
Sep 14 08:21:11.292949 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.292845    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf" event={"ID":"c58ec321-5017-43e7-adbe-107cd0f3b7c4","Type":"ContainerDied","Data":"2bd6ebe2b104f8dbaafd0eaf8b7d06c8fe5c2867a61c4b2b8b0dce89d9e01051"}
Sep 14 08:21:11.292949 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.292914    2562 scope.go:117] "RemoveContainer" containerID="c05c73a8b9ad9c1f4834e36e6e0904191a2bd203d260985fd36d5de58faeeb28"
Sep 14 08:21:11.311146 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:21:11.311119    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c05c73a8b9ad9c1f4834e36e6e0904191a2bd203d260985fd36d5de58faeeb28\": container with ID starting with c05c73a8b9ad9c1f4834e36e6e0904191a2bd203d260985fd36d5de58faeeb28 not found: ID does not exist" containerID="c05c73a8b9ad9c1f4834e36e6e0904191a2bd203d260985fd36d5de58faeeb28"
Sep 14 08:21:11.315280 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.315254    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:21:11.350023 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.349940    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-workload-socket\") pod \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") "
Sep 14 08:21:11.350023 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.349995    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-podinfo\") pod \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") "
Sep 14 08:21:11.350238 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.350033    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-workload-certs\") pod \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") "
Sep 14 08:21:11.350238 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.350058    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-data\") pod \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") "
Sep 14 08:21:11.350238 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.350083    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-envoy\") pod \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") "
Sep 14 08:21:11.350238 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.350111    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istiod-ca-cert\") pod \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") "
Sep 14 08:21:11.350238 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.350188    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c58ec321-5017-43e7-adbe-107cd0f3b7c4" (UID: "c58ec321-5017-43e7-adbe-107cd0f3b7c4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:21:11.350238 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.350210    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-token\") pod \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") "
Sep 14 08:21:11.350569 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.350318    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-credential-socket\") pod \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") "
Sep 14 08:21:11.350569 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.350329    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-data" (OuterVolumeSpecName: "istio-data") pod "c58ec321-5017-43e7-adbe-107cd0f3b7c4" (UID: "c58ec321-5017-43e7-adbe-107cd0f3b7c4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:21:11.350569 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.350346    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c58ec321-5017-43e7-adbe-107cd0f3b7c4" (UID: "c58ec321-5017-43e7-adbe-107cd0f3b7c4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:21:11.350569 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.350352    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7kss\" (UniqueName: \"kubernetes.io/projected/c58ec321-5017-43e7-adbe-107cd0f3b7c4-kube-api-access-w7kss\") pod \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\" (UID: \"c58ec321-5017-43e7-adbe-107cd0f3b7c4\") "
Sep 14 08:21:11.350569 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.350547    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c58ec321-5017-43e7-adbe-107cd0f3b7c4" (UID: "c58ec321-5017-43e7-adbe-107cd0f3b7c4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:21:11.350816 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.350742    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-credential-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:21:11.350816 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.350763    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-workload-socket\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:21:11.350816 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.350777    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-workload-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:21:11.350816 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.350791    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-data\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:21:11.350994 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.350843    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c58ec321-5017-43e7-adbe-107cd0f3b7c4" (UID: "c58ec321-5017-43e7-adbe-107cd0f3b7c4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:21:11.352667 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.352645    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-token" (OuterVolumeSpecName: "istio-token") pod "c58ec321-5017-43e7-adbe-107cd0f3b7c4" (UID: "c58ec321-5017-43e7-adbe-107cd0f3b7c4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:21:11.352937 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.352914    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c58ec321-5017-43e7-adbe-107cd0f3b7c4" (UID: "c58ec321-5017-43e7-adbe-107cd0f3b7c4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:21:11.353005 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.352914    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c58ec321-5017-43e7-adbe-107cd0f3b7c4" (UID: "c58ec321-5017-43e7-adbe-107cd0f3b7c4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:21:11.353058 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.352998    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c58ec321-5017-43e7-adbe-107cd0f3b7c4-kube-api-access-w7kss" (OuterVolumeSpecName: "kube-api-access-w7kss") pod "c58ec321-5017-43e7-adbe-107cd0f3b7c4" (UID: "c58ec321-5017-43e7-adbe-107cd0f3b7c4"). InnerVolumeSpecName "kube-api-access-w7kss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:21:11.451744 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.451707    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-podinfo\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:21:11.451744 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.451737    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-envoy\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:21:11.451744 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.451746    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istiod-ca-cert\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:21:11.451744 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.451754    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c58ec321-5017-43e7-adbe-107cd0f3b7c4-istio-token\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:21:11.452013 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:11.451763    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7kss\" (UniqueName: \"kubernetes.io/projected/c58ec321-5017-43e7-adbe-107cd0f3b7c4-kube-api-access-w7kss\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:21:12.298597 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:12.298568    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"
Sep 14 08:21:12.298597 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:12.298592    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf" event={"ID":"c58ec321-5017-43e7-adbe-107cd0f3b7c4","Type":"ContainerDied","Data":"66405745b9d0d910a6ea5662c0a771d4dacb5ba76930d594a4222a54a484e062"}
Sep 14 08:21:12.299055 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:12.298650    2562 scope.go:117] "RemoveContainer" containerID="2bd6ebe2b104f8dbaafd0eaf8b7d06c8fe5c2867a61c4b2b8b0dce89d9e01051"
Sep 14 08:21:12.320782 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:12.320752    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"]
Sep 14 08:21:12.325765 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:12.325736    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-ea9d35-openshift-default-646fd4b4b8-nh9lf"]
Sep 14 08:21:14.112642 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:14.112608    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c58ec321-5017-43e7-adbe-107cd0f3b7c4" path="/var/lib/kubelet/pods/c58ec321-5017-43e7-adbe-107cd0f3b7c4/volumes"
Sep 14 08:21:18.346649 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:18.346616    2562 scope.go:117] "RemoveContainer" containerID="d605e9684839241b5d5a2e0a346a4bc16515ae9370e1165921cf0a09c8f0acc9"
Sep 14 08:21:50.282721 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:50.282667    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:50.337073 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:50.337036    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:21:58.225198 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:21:58.225161    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:22:31.395809 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:22:31.395769    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:22:32.390957 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:22:32.390924    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:22:33.978261 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:22:33.978226    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:22:52.397077 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:22:52.396988    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:22:58.102402 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:22:58.102367    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:23:05.797704 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:23:05.797663    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:23:13.103008 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:23:13.102969    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:23:20.493440 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:23:20.493403    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:23:35.409884 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:23:35.409846    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:23:43.095384 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:23:43.095349    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:23:53.516148 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:23:53.516109    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:23:53.616402 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:23:53.616365    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:24:01.585727 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:24:01.585687    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:24:29.841932 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:24:29.841838    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:25:12.800132 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:25:12.800094    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:25:20.800446 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:25:20.800410    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:25:27.001932 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:25:27.001890    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:25:29.403874 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:25:29.403837    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:25:32.906607 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:25:32.906564    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:25:44.508435 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:25:44.508352    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:25:47.401694 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:25:47.401651    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:26:13.699616 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:13.699578    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:26:14.600874 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:14.600831    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:26:17.006992 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:17.006954    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:26:26.102331 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:26.102294    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:26:31.724736 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.724696    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"]
Sep 14 08:26:31.725121 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.725083    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c58ec321-5017-43e7-adbe-107cd0f3b7c4" containerName="istio-proxy"
Sep 14 08:26:31.725121 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.725095    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c58ec321-5017-43e7-adbe-107cd0f3b7c4" containerName="istio-proxy"
Sep 14 08:26:31.725121 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.725113    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10943269-128b-4c50-82d8-e70c44c3a70f" containerName="istio-proxy"
Sep 14 08:26:31.725121 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.725119    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="10943269-128b-4c50-82d8-e70c44c3a70f" containerName="istio-proxy"
Sep 14 08:26:31.725250 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.725135    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10943269-128b-4c50-82d8-e70c44c3a70f" containerName="istio-proxy"
Sep 14 08:26:31.725250 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.725141    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="10943269-128b-4c50-82d8-e70c44c3a70f" containerName="istio-proxy"
Sep 14 08:26:31.725250 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.725148    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc69bf26-aab4-480b-b0a5-ad4beaa7606e" containerName="revoke-keys"
Sep 14 08:26:31.725250 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.725153    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc69bf26-aab4-480b-b0a5-ad4beaa7606e" containerName="revoke-keys"
Sep 14 08:26:31.725250 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.725206    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc69bf26-aab4-480b-b0a5-ad4beaa7606e" containerName="revoke-keys"
Sep 14 08:26:31.725250 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.725213    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c58ec321-5017-43e7-adbe-107cd0f3b7c4" containerName="istio-proxy"
Sep 14 08:26:31.725250 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.725219    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c58ec321-5017-43e7-adbe-107cd0f3b7c4" containerName="istio-proxy"
Sep 14 08:26:31.725250 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.725225    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="10943269-128b-4c50-82d8-e70c44c3a70f" containerName="istio-proxy"
Sep 14 08:26:31.728297 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.728266    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"
Sep 14 08:26:31.731273 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.731251    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 14 08:26:31.731408 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.731268    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 08:26:31.731408 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.731312    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 08:26:31.731408 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.731274    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-g2t2q\""
Sep 14 08:26:31.731408 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.731340    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 08:26:31.737623 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.737594    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"]
Sep 14 08:26:31.841022 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.840981    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjph2\" (UniqueName: \"kubernetes.io/projected/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-kube-api-access-pjph2\") pod \"maas-api-765c9ffcd4-l4vwk\" (UID: \"d293ec2f-6bc5-4008-87e4-f6bde91acbf0\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"
Sep 14 08:26:31.841226 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.841045    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-openshift-service-ca\") pod \"maas-api-765c9ffcd4-l4vwk\" (UID: \"d293ec2f-6bc5-4008-87e4-f6bde91acbf0\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"
Sep 14 08:26:31.841226 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.841093    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-metrics-certs\") pod \"maas-api-765c9ffcd4-l4vwk\" (UID: \"d293ec2f-6bc5-4008-87e4-f6bde91acbf0\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"
Sep 14 08:26:31.841226 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.841171    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-maas-api-tls\") pod \"maas-api-765c9ffcd4-l4vwk\" (UID: \"d293ec2f-6bc5-4008-87e4-f6bde91acbf0\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"
Sep 14 08:26:31.942056 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.942016    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-openshift-service-ca\") pod \"maas-api-765c9ffcd4-l4vwk\" (UID: \"d293ec2f-6bc5-4008-87e4-f6bde91acbf0\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"
Sep 14 08:26:31.942249 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.942074    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-metrics-certs\") pod \"maas-api-765c9ffcd4-l4vwk\" (UID: \"d293ec2f-6bc5-4008-87e4-f6bde91acbf0\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"
Sep 14 08:26:31.942249 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.942117    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-maas-api-tls\") pod \"maas-api-765c9ffcd4-l4vwk\" (UID: \"d293ec2f-6bc5-4008-87e4-f6bde91acbf0\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"
Sep 14 08:26:31.942249 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.942171    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjph2\" (UniqueName: \"kubernetes.io/projected/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-kube-api-access-pjph2\") pod \"maas-api-765c9ffcd4-l4vwk\" (UID: \"d293ec2f-6bc5-4008-87e4-f6bde91acbf0\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"
Sep 14 08:26:31.942748 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.942722    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-openshift-service-ca\") pod \"maas-api-765c9ffcd4-l4vwk\" (UID: \"d293ec2f-6bc5-4008-87e4-f6bde91acbf0\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"
Sep 14 08:26:31.944562 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.944540    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-metrics-certs\") pod \"maas-api-765c9ffcd4-l4vwk\" (UID: \"d293ec2f-6bc5-4008-87e4-f6bde91acbf0\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"
Sep 14 08:26:31.944697 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.944573    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-maas-api-tls\") pod \"maas-api-765c9ffcd4-l4vwk\" (UID: \"d293ec2f-6bc5-4008-87e4-f6bde91acbf0\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"
Sep 14 08:26:31.950564 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:31.950541    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjph2\" (UniqueName: \"kubernetes.io/projected/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-kube-api-access-pjph2\") pod \"maas-api-765c9ffcd4-l4vwk\" (UID: \"d293ec2f-6bc5-4008-87e4-f6bde91acbf0\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"
Sep 14 08:26:32.040162 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:32.040068    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"
Sep 14 08:26:32.208147 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:32.208111    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:26:32.313419 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:32.313391    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"]
Sep 14 08:26:32.315800 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:26:32.315773    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd293ec2f_6bc5_4008_87e4_f6bde91acbf0.slice/crio-5db25788611783af344cd65ead87544a9c6cd383e75c89f49f7b0d2c87f494cd WatchSource:0}: Error finding container 5db25788611783af344cd65ead87544a9c6cd383e75c89f49f7b0d2c87f494cd: Status 404 returned error can't find the container with id 5db25788611783af344cd65ead87544a9c6cd383e75c89f49f7b0d2c87f494cd
Sep 14 08:26:32.317649 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:32.317631    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 08:26:32.828711 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:32.828663    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk" event={"ID":"d293ec2f-6bc5-4008-87e4-f6bde91acbf0","Type":"ContainerStarted","Data":"5db25788611783af344cd65ead87544a9c6cd383e75c89f49f7b0d2c87f494cd"}
Sep 14 08:26:33.834927 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:33.834892    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk" event={"ID":"d293ec2f-6bc5-4008-87e4-f6bde91acbf0","Type":"ContainerStarted","Data":"c78cb632cbc9e0a60e02eb97d156d4bea0a0e3c488a9b2f190cdcef3bbd8bef3"}
Sep 14 08:26:33.835316 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:33.834997    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"
Sep 14 08:26:33.854702 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:33.854653    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk" podStartSLOduration=2.291867131 podStartE2EDuration="2.854639588s" totalImagesPullingTime="562.772457ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:26:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:26:33.853876774 +0000 UTC m=+2296.310326768" watchObservedRunningTime="2026-09-14 08:26:33.854639588 +0000 UTC m=+2296.311089575"
Sep 14 08:26:39.299671 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.299617    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:26:39.418220 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.418187    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"]
Sep 14 08:26:39.418423 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.418370    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk" podUID="d293ec2f-6bc5-4008-87e4-f6bde91acbf0" containerName="maas-api" containerID="cri-o://c78cb632cbc9e0a60e02eb97d156d4bea0a0e3c488a9b2f190cdcef3bbd8bef3" gracePeriod=30
Sep 14 08:26:39.423822 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.423798    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"
Sep 14 08:26:39.449882 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.449850    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8456c7bcb-ccqcs"]
Sep 14 08:26:39.450208 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.450196    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c58ec321-5017-43e7-adbe-107cd0f3b7c4" containerName="istio-proxy"
Sep 14 08:26:39.450256 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.450209    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c58ec321-5017-43e7-adbe-107cd0f3b7c4" containerName="istio-proxy"
Sep 14 08:26:39.450292 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.450275    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="10943269-128b-4c50-82d8-e70c44c3a70f" containerName="istio-proxy"
Sep 14 08:26:39.453073 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.453051    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8456c7bcb-ccqcs"
Sep 14 08:26:39.463559 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.463526    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8456c7bcb-ccqcs"]
Sep 14 08:26:39.609088 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.609051    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2bnx\" (UniqueName: \"kubernetes.io/projected/4629297a-93a8-4f6c-9fab-d70bd20cc0a5-kube-api-access-q2bnx\") pod \"maas-api-8456c7bcb-ccqcs\" (UID: \"4629297a-93a8-4f6c-9fab-d70bd20cc0a5\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-ccqcs"
Sep 14 08:26:39.609266 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.609187    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4629297a-93a8-4f6c-9fab-d70bd20cc0a5-openshift-service-ca\") pod \"maas-api-8456c7bcb-ccqcs\" (UID: \"4629297a-93a8-4f6c-9fab-d70bd20cc0a5\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-ccqcs"
Sep 14 08:26:39.609266 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.609222    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4629297a-93a8-4f6c-9fab-d70bd20cc0a5-maas-api-tls\") pod \"maas-api-8456c7bcb-ccqcs\" (UID: \"4629297a-93a8-4f6c-9fab-d70bd20cc0a5\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-ccqcs"
Sep 14 08:26:39.609266 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.609260    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4629297a-93a8-4f6c-9fab-d70bd20cc0a5-metrics-certs\") pod \"maas-api-8456c7bcb-ccqcs\" (UID: \"4629297a-93a8-4f6c-9fab-d70bd20cc0a5\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-ccqcs"
Sep 14 08:26:39.669154 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.669127    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"
Sep 14 08:26:39.710611 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.710569    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2bnx\" (UniqueName: \"kubernetes.io/projected/4629297a-93a8-4f6c-9fab-d70bd20cc0a5-kube-api-access-q2bnx\") pod \"maas-api-8456c7bcb-ccqcs\" (UID: \"4629297a-93a8-4f6c-9fab-d70bd20cc0a5\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-ccqcs"
Sep 14 08:26:39.710798 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.710709    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4629297a-93a8-4f6c-9fab-d70bd20cc0a5-openshift-service-ca\") pod \"maas-api-8456c7bcb-ccqcs\" (UID: \"4629297a-93a8-4f6c-9fab-d70bd20cc0a5\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-ccqcs"
Sep 14 08:26:39.710798 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.710749    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4629297a-93a8-4f6c-9fab-d70bd20cc0a5-maas-api-tls\") pod \"maas-api-8456c7bcb-ccqcs\" (UID: \"4629297a-93a8-4f6c-9fab-d70bd20cc0a5\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-ccqcs"
Sep 14 08:26:39.710798 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.710791    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4629297a-93a8-4f6c-9fab-d70bd20cc0a5-metrics-certs\") pod \"maas-api-8456c7bcb-ccqcs\" (UID: \"4629297a-93a8-4f6c-9fab-d70bd20cc0a5\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-ccqcs"
Sep 14 08:26:39.711776 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.711744    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4629297a-93a8-4f6c-9fab-d70bd20cc0a5-openshift-service-ca\") pod \"maas-api-8456c7bcb-ccqcs\" (UID: \"4629297a-93a8-4f6c-9fab-d70bd20cc0a5\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-ccqcs"
Sep 14 08:26:39.713940 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.713903    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4629297a-93a8-4f6c-9fab-d70bd20cc0a5-metrics-certs\") pod \"maas-api-8456c7bcb-ccqcs\" (UID: \"4629297a-93a8-4f6c-9fab-d70bd20cc0a5\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-ccqcs"
Sep 14 08:26:39.714317 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.714288    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4629297a-93a8-4f6c-9fab-d70bd20cc0a5-maas-api-tls\") pod \"maas-api-8456c7bcb-ccqcs\" (UID: \"4629297a-93a8-4f6c-9fab-d70bd20cc0a5\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-ccqcs"
Sep 14 08:26:39.718949 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.718924    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2bnx\" (UniqueName: \"kubernetes.io/projected/4629297a-93a8-4f6c-9fab-d70bd20cc0a5-kube-api-access-q2bnx\") pod \"maas-api-8456c7bcb-ccqcs\" (UID: \"4629297a-93a8-4f6c-9fab-d70bd20cc0a5\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-ccqcs"
Sep 14 08:26:39.802603 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.802562    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8456c7bcb-ccqcs"
Sep 14 08:26:39.811980 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.811953    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-metrics-certs\") pod \"d293ec2f-6bc5-4008-87e4-f6bde91acbf0\" (UID: \"d293ec2f-6bc5-4008-87e4-f6bde91acbf0\") "
Sep 14 08:26:39.812060 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.812007    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-openshift-service-ca\") pod \"d293ec2f-6bc5-4008-87e4-f6bde91acbf0\" (UID: \"d293ec2f-6bc5-4008-87e4-f6bde91acbf0\") "
Sep 14 08:26:39.812060 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.812035    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjph2\" (UniqueName: \"kubernetes.io/projected/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-kube-api-access-pjph2\") pod \"d293ec2f-6bc5-4008-87e4-f6bde91acbf0\" (UID: \"d293ec2f-6bc5-4008-87e4-f6bde91acbf0\") "
Sep 14 08:26:39.812148 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.812082    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-maas-api-tls\") pod \"d293ec2f-6bc5-4008-87e4-f6bde91acbf0\" (UID: \"d293ec2f-6bc5-4008-87e4-f6bde91acbf0\") "
Sep 14 08:26:39.812376 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.812344    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d293ec2f-6bc5-4008-87e4-f6bde91acbf0" (UID: "d293ec2f-6bc5-4008-87e4-f6bde91acbf0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:26:39.814757 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.814728    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-kube-api-access-pjph2" (OuterVolumeSpecName: "kube-api-access-pjph2") pod "d293ec2f-6bc5-4008-87e4-f6bde91acbf0" (UID: "d293ec2f-6bc5-4008-87e4-f6bde91acbf0"). InnerVolumeSpecName "kube-api-access-pjph2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:26:39.814862 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.814784    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d293ec2f-6bc5-4008-87e4-f6bde91acbf0" (UID: "d293ec2f-6bc5-4008-87e4-f6bde91acbf0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:26:39.815266 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.815241    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d293ec2f-6bc5-4008-87e4-f6bde91acbf0" (UID: "d293ec2f-6bc5-4008-87e4-f6bde91acbf0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:26:39.868587 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.868551    2562 generic.go:356] "Generic (PLEG): container finished" podID="d293ec2f-6bc5-4008-87e4-f6bde91acbf0" containerID="c78cb632cbc9e0a60e02eb97d156d4bea0a0e3c488a9b2f190cdcef3bbd8bef3" exitCode=0
Sep 14 08:26:39.868750 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.868608    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk" event={"ID":"d293ec2f-6bc5-4008-87e4-f6bde91acbf0","Type":"ContainerDied","Data":"c78cb632cbc9e0a60e02eb97d156d4bea0a0e3c488a9b2f190cdcef3bbd8bef3"}
Sep 14 08:26:39.868750 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.868693    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk" event={"ID":"d293ec2f-6bc5-4008-87e4-f6bde91acbf0","Type":"ContainerDied","Data":"5db25788611783af344cd65ead87544a9c6cd383e75c89f49f7b0d2c87f494cd"}
Sep 14 08:26:39.868750 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.868724    2562 scope.go:117] "RemoveContainer" containerID="c78cb632cbc9e0a60e02eb97d156d4bea0a0e3c488a9b2f190cdcef3bbd8bef3"
Sep 14 08:26:39.868928 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.868884    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"
Sep 14 08:26:39.881115 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.881094    2562 scope.go:117] "RemoveContainer" containerID="c78cb632cbc9e0a60e02eb97d156d4bea0a0e3c488a9b2f190cdcef3bbd8bef3"
Sep 14 08:26:39.881770 ip-10-0-138-28 kubenswrapper[2562]: E0914 08:26:39.881730    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c78cb632cbc9e0a60e02eb97d156d4bea0a0e3c488a9b2f190cdcef3bbd8bef3\": container with ID starting with c78cb632cbc9e0a60e02eb97d156d4bea0a0e3c488a9b2f190cdcef3bbd8bef3 not found: ID does not exist" containerID="c78cb632cbc9e0a60e02eb97d156d4bea0a0e3c488a9b2f190cdcef3bbd8bef3"
Sep 14 08:26:39.881883 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.881767    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c78cb632cbc9e0a60e02eb97d156d4bea0a0e3c488a9b2f190cdcef3bbd8bef3"} err="failed to get container status \"c78cb632cbc9e0a60e02eb97d156d4bea0a0e3c488a9b2f190cdcef3bbd8bef3\": rpc error: code = NotFound desc = could not find container \"c78cb632cbc9e0a60e02eb97d156d4bea0a0e3c488a9b2f190cdcef3bbd8bef3\": container with ID starting with c78cb632cbc9e0a60e02eb97d156d4bea0a0e3c488a9b2f190cdcef3bbd8bef3 not found: ID does not exist"
Sep 14 08:26:39.894324 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.894294    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"]
Sep 14 08:26:39.897562 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.897534    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-765c9ffcd4-l4vwk"]
Sep 14 08:26:39.913050 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.913017    2562 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-openshift-service-ca\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:26:39.913050 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.913048    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjph2\" (UniqueName: \"kubernetes.io/projected/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-kube-api-access-pjph2\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:26:39.913205 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.913064    2562 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-maas-api-tls\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:26:39.913205 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:39.913076    2562 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d293ec2f-6bc5-4008-87e4-f6bde91acbf0-metrics-certs\") on node \"ip-10-0-138-28.ec2.internal\" DevicePath \"\""
Sep 14 08:26:40.080010 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:40.079814    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8456c7bcb-ccqcs"]
Sep 14 08:26:40.082769 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:26:40.082735    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4629297a_93a8_4f6c_9fab_d70bd20cc0a5.slice/crio-6531fc2b4bba7e46485afc3df599c2a01ef96360b104719ba6a500a6619234e5 WatchSource:0}: Error finding container 6531fc2b4bba7e46485afc3df599c2a01ef96360b104719ba6a500a6619234e5: Status 404 returned error can't find the container with id 6531fc2b4bba7e46485afc3df599c2a01ef96360b104719ba6a500a6619234e5
Sep 14 08:26:40.113761 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:40.113677    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d293ec2f-6bc5-4008-87e4-f6bde91acbf0" path="/var/lib/kubelet/pods/d293ec2f-6bc5-4008-87e4-f6bde91acbf0/volumes"
Sep 14 08:26:40.875826 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:40.875786    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8456c7bcb-ccqcs" event={"ID":"4629297a-93a8-4f6c-9fab-d70bd20cc0a5","Type":"ContainerStarted","Data":"d48921c499f961b6f9ea0389e2bb2e511caa623ad6dabea1d44c855e24b32155"}
Sep 14 08:26:40.876291 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:40.875862    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8456c7bcb-ccqcs"
Sep 14 08:26:40.876291 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:40.875887    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8456c7bcb-ccqcs" event={"ID":"4629297a-93a8-4f6c-9fab-d70bd20cc0a5","Type":"ContainerStarted","Data":"6531fc2b4bba7e46485afc3df599c2a01ef96360b104719ba6a500a6619234e5"}
Sep 14 08:26:40.897085 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:40.897031    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8456c7bcb-ccqcs" podStartSLOduration=1.5746342260000001 podStartE2EDuration="1.897011875s" totalImagesPullingTime="322.377649ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:26:40.895944297 +0000 UTC m=+2303.352394292" watchObservedRunningTime="2026-09-14 08:26:40.897011875 +0000 UTC m=+2303.353461911"
Sep 14 08:26:46.114940 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:46.114910    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:26:46.887401 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:46.887371    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8456c7bcb-ccqcs"
Sep 14 08:26:51.807191 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:51.807151    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:26:58.702516 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:26:58.702468    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-rv6tc"]
Sep 14 08:27:07.525560 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:07.525523    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-hxw5j_db58eff4-ee8e-497f-a4e7-3e76297e5a7f/manager/0.log"
Sep 14 08:27:07.900046 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:07.899961    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-8mnfx_646ed64e-44c7-4bce-a9fb-0a8ff74e6774/server/0.log"
Sep 14 08:27:08.392568 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:08.392539    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c4c5b687-twcct_1962415f-77ed-420f-afa0-38b3947646ef/copy-manifests/0.log"
Sep 14 08:27:08.401884 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:08.401853    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c4c5b687-twcct_1962415f-77ed-420f-afa0-38b3947646ef/manager/0.log"
Sep 14 08:27:08.741108 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:08.741076    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-8456c7bcb-ccqcs_4629297a-93a8-4f6c-9fab-d70bd20cc0a5/maas-api/0.log"
Sep 14 08:27:09.078090 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:09.078007    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29822895-6965l_5986f5dd-78bb-4bcf-968d-d0ff466c4443/cleanup/0.log"
Sep 14 08:27:10.095424 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:10.095382    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-g7ddw_65e6ce0b-ec50-46ef-bd96-f3dc3fb1bdde/manager/0.log"
Sep 14 08:27:10.209973 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:10.209937    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-klrkj_a5d25397-2d62-4dc7-93be-60428216e944/manager/0.log"
Sep 14 08:27:10.566233 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:10.566202    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-rv6tc_51555902-c387-45cb-ac73-3f43dc26962a/limitador/0.log"
Sep 14 08:27:11.042772 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:11.042740    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf_101c9562-855a-4776-9df4-c40aaef6eb5a/istio-proxy/0.log"
Sep 14 08:27:11.505238 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:11.505186    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-zdnqg_5f6db21f-387c-48d5-a12f-9f809fbd87d2/istio-proxy/1.log"
Sep 14 08:27:12.449240 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:12.449202    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-266jz_87434236-fa77-40b1-a197-b0f4ca8f22bc/main/0.log"
Sep 14 08:27:12.670624 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:12.670585    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68ftrs47_5400c3d5-1766-439c-947d-5da15cf447ee/main/0.log"
Sep 14 08:27:12.779257 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:12.779171    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-kqcnm_c95803a4-5a40-40e5-bee2-22012d82167f/main/0.log"
Sep 14 08:27:18.413970 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:18.413937    2562 scope.go:117] "RemoveContainer" containerID="e1497e3bff09e023d9c7ecb9577d736235e9f0a1af5c59f850a6cac8e0067000"
Sep 14 08:27:18.665247 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:18.665164    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:27:18.733838 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:18.733802    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log" limit=1024
Sep 14 08:27:22.068335 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:22.068298    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xbb69_4b4a6526-1631-4af9-9a20-cf80f53d3eac/global-pull-secret-syncer/0.log"
Sep 14 08:27:22.264715 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:22.264678    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-b2sjm_d4c06442-0a48-45fb-a418-e51fa6a4cea4/konnectivity-agent/0.log"
Sep 14 08:27:22.552365 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:22.552333    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-28.ec2.internal_ef4817a2551205637f03cf112a16afcb/haproxy/0.log"
Sep 14 08:27:27.186540 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:27.186491    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-g7ddw_65e6ce0b-ec50-46ef-bd96-f3dc3fb1bdde/manager/0.log"
Sep 14 08:27:27.234389 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:27.234364    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-klrkj_a5d25397-2d62-4dc7-93be-60428216e944/manager/0.log"
Sep 14 08:27:27.365561 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:27.365527    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-rv6tc_51555902-c387-45cb-ac73-3f43dc26962a/limitador/0.log"
Sep 14 08:27:29.300609 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:29.300582    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9sqqx_92a3911a-36bc-4793-bfad-d598640f14ef/node-exporter/0.log"
Sep 14 08:27:29.325753 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:29.325726    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9sqqx_92a3911a-36bc-4793-bfad-d598640f14ef/kube-rbac-proxy/0.log"
Sep 14 08:27:29.344456 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:29.344429    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9sqqx_92a3911a-36bc-4793-bfad-d598640f14ef/init-textfile/0.log"
Sep 14 08:27:29.530826 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:29.530754    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5d75856f-ee1d-42ac-acfc-b57ec2965a2b/prometheus/0.log"
Sep 14 08:27:29.549716 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:29.549684    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5d75856f-ee1d-42ac-acfc-b57ec2965a2b/config-reloader/0.log"
Sep 14 08:27:29.569138 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:29.569113    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5d75856f-ee1d-42ac-acfc-b57ec2965a2b/thanos-sidecar/0.log"
Sep 14 08:27:29.588465 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:29.588442    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5d75856f-ee1d-42ac-acfc-b57ec2965a2b/kube-rbac-proxy-web/0.log"
Sep 14 08:27:29.610635 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:29.610605    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5d75856f-ee1d-42ac-acfc-b57ec2965a2b/kube-rbac-proxy/0.log"
Sep 14 08:27:29.632620 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:29.632594    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5d75856f-ee1d-42ac-acfc-b57ec2965a2b/kube-rbac-proxy-thanos/0.log"
Sep 14 08:27:29.656532 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:29.656488    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5d75856f-ee1d-42ac-acfc-b57ec2965a2b/init-config-reloader/0.log"
Sep 14 08:27:29.849581 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:29.849486    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c7dc76f84-gdmhf_173d0d44-527b-47d0-99e2-3ffb1058ba31/thanos-query/0.log"
Sep 14 08:27:29.870192 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:29.870157    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c7dc76f84-gdmhf_173d0d44-527b-47d0-99e2-3ffb1058ba31/kube-rbac-proxy-web/0.log"
Sep 14 08:27:29.890147 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:29.890118    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c7dc76f84-gdmhf_173d0d44-527b-47d0-99e2-3ffb1058ba31/kube-rbac-proxy/0.log"
Sep 14 08:27:29.914912 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:29.914884    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c7dc76f84-gdmhf_173d0d44-527b-47d0-99e2-3ffb1058ba31/prom-label-proxy/0.log"
Sep 14 08:27:29.939080 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:29.939052    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c7dc76f84-gdmhf_173d0d44-527b-47d0-99e2-3ffb1058ba31/kube-rbac-proxy-rules/0.log"
Sep 14 08:27:29.959194 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:29.959155    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c7dc76f84-gdmhf_173d0d44-527b-47d0-99e2-3ffb1058ba31/kube-rbac-proxy-metrics/0.log"
Sep 14 08:27:31.979695 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:31.979650    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s"]
Sep 14 08:27:31.980117 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:31.980069    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d293ec2f-6bc5-4008-87e4-f6bde91acbf0" containerName="maas-api"
Sep 14 08:27:31.980117 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:31.980082    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d293ec2f-6bc5-4008-87e4-f6bde91acbf0" containerName="maas-api"
Sep 14 08:27:31.980196 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:31.980145    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d293ec2f-6bc5-4008-87e4-f6bde91acbf0" containerName="maas-api"
Sep 14 08:27:31.983422 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:31.983398    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s"
Sep 14 08:27:31.985752 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:31.985714    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ds52d\"/\"openshift-service-ca.crt\""
Sep 14 08:27:31.985752 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:31.985729    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ds52d\"/\"default-dockercfg-s6bnm\""
Sep 14 08:27:31.986146 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:31.986126    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ds52d\"/\"kube-root-ca.crt\""
Sep 14 08:27:31.993692 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:31.993665    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s"]
Sep 14 08:27:32.181794 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:32.181755    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8946\" (UniqueName: \"kubernetes.io/projected/a42b5742-782d-4847-bbd2-250630712722-kube-api-access-j8946\") pod \"perf-node-gather-daemonset-vr64s\" (UID: \"a42b5742-782d-4847-bbd2-250630712722\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s"
Sep 14 08:27:32.181986 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:32.181821    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a42b5742-782d-4847-bbd2-250630712722-proc\") pod \"perf-node-gather-daemonset-vr64s\" (UID: \"a42b5742-782d-4847-bbd2-250630712722\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s"
Sep 14 08:27:32.181986 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:32.181845    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a42b5742-782d-4847-bbd2-250630712722-sys\") pod \"perf-node-gather-daemonset-vr64s\" (UID: \"a42b5742-782d-4847-bbd2-250630712722\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s"
Sep 14 08:27:32.181986 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:32.181884    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a42b5742-782d-4847-bbd2-250630712722-lib-modules\") pod \"perf-node-gather-daemonset-vr64s\" (UID: \"a42b5742-782d-4847-bbd2-250630712722\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s"
Sep 14 08:27:32.182139 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:32.182003    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a42b5742-782d-4847-bbd2-250630712722-podres\") pod \"perf-node-gather-daemonset-vr64s\" (UID: \"a42b5742-782d-4847-bbd2-250630712722\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s"
Sep 14 08:27:32.283632 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:32.283534    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a42b5742-782d-4847-bbd2-250630712722-lib-modules\") pod \"perf-node-gather-daemonset-vr64s\" (UID: \"a42b5742-782d-4847-bbd2-250630712722\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s"
Sep 14 08:27:32.283632 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:32.283590    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a42b5742-782d-4847-bbd2-250630712722-podres\") pod \"perf-node-gather-daemonset-vr64s\" (UID: \"a42b5742-782d-4847-bbd2-250630712722\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s"
Sep 14 08:27:32.283868 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:32.283639    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8946\" (UniqueName: \"kubernetes.io/projected/a42b5742-782d-4847-bbd2-250630712722-kube-api-access-j8946\") pod \"perf-node-gather-daemonset-vr64s\" (UID: \"a42b5742-782d-4847-bbd2-250630712722\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s"
Sep 14 08:27:32.283868 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:32.283691    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a42b5742-782d-4847-bbd2-250630712722-proc\") pod \"perf-node-gather-daemonset-vr64s\" (UID: \"a42b5742-782d-4847-bbd2-250630712722\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s"
Sep 14 08:27:32.283868 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:32.283714    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a42b5742-782d-4847-bbd2-250630712722-sys\") pod \"perf-node-gather-daemonset-vr64s\" (UID: \"a42b5742-782d-4847-bbd2-250630712722\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s"
Sep 14 08:27:32.283868 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:32.283749    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a42b5742-782d-4847-bbd2-250630712722-lib-modules\") pod \"perf-node-gather-daemonset-vr64s\" (UID: \"a42b5742-782d-4847-bbd2-250630712722\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s"
Sep 14 08:27:32.283868 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:32.283775    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a42b5742-782d-4847-bbd2-250630712722-podres\") pod \"perf-node-gather-daemonset-vr64s\" (UID: \"a42b5742-782d-4847-bbd2-250630712722\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s"
Sep 14 08:27:32.283868 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:32.283818    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a42b5742-782d-4847-bbd2-250630712722-proc\") pod \"perf-node-gather-daemonset-vr64s\" (UID: \"a42b5742-782d-4847-bbd2-250630712722\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s"
Sep 14 08:27:32.283868 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:32.283824    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a42b5742-782d-4847-bbd2-250630712722-sys\") pod \"perf-node-gather-daemonset-vr64s\" (UID: \"a42b5742-782d-4847-bbd2-250630712722\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s"
Sep 14 08:27:32.292372 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:32.292338    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8946\" (UniqueName: \"kubernetes.io/projected/a42b5742-782d-4847-bbd2-250630712722-kube-api-access-j8946\") pod \"perf-node-gather-daemonset-vr64s\" (UID: \"a42b5742-782d-4847-bbd2-250630712722\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s"
Sep 14 08:27:32.296189 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:32.296160    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s"
Sep 14 08:27:32.567626 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:32.567524    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s"]
Sep 14 08:27:32.572080 ip-10-0-138-28 kubenswrapper[2562]: W0914 08:27:32.572047    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda42b5742_782d_4847_bbd2_250630712722.slice/crio-089f53de683e284c7e208281ed603844fe2561d67dee7290ecc6d4551a129933 WatchSource:0}: Error finding container 089f53de683e284c7e208281ed603844fe2561d67dee7290ecc6d4551a129933: Status 404 returned error can't find the container with id 089f53de683e284c7e208281ed603844fe2561d67dee7290ecc6d4551a129933
Sep 14 08:27:33.127276 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:33.127237    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s" event={"ID":"a42b5742-782d-4847-bbd2-250630712722","Type":"ContainerStarted","Data":"55cfc701da21562d9e065357eca0c0e1cb867328dae04adabf9d4617e1c8d30b"}
Sep 14 08:27:33.127710 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:33.127291    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s" event={"ID":"a42b5742-782d-4847-bbd2-250630712722","Type":"ContainerStarted","Data":"089f53de683e284c7e208281ed603844fe2561d67dee7290ecc6d4551a129933"}
Sep 14 08:27:33.127710 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:33.127357    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s"
Sep 14 08:27:33.151710 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:33.151513    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s" podStartSLOduration=2.1514389019999998 podStartE2EDuration="2.151438902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 08:27:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:27:33.143923315 +0000 UTC m=+2355.600373308" watchObservedRunningTime="2026-09-14 08:27:33.151438902 +0000 UTC m=+2355.607888896"
Sep 14 08:27:34.453907 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:34.453871    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mfdkf_649d1d31-0095-4b78-89e6-cecf97d8d72a/dns/0.log"
Sep 14 08:27:34.486043 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:34.486013    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mfdkf_649d1d31-0095-4b78-89e6-cecf97d8d72a/kube-rbac-proxy/0.log"
Sep 14 08:27:34.577532 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:34.577481    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6xvpb_25a3ecfc-dade-4e7c-8347-ce9f629d1f37/dns-node-resolver/0.log"
Sep 14 08:27:35.259204 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:35.259174    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pdfcv_9594803d-c72d-4f5a-9b8d-e78d40e17d92/node-ca/0.log"
Sep 14 08:27:36.266175 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:36.266140    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf_101c9562-855a-4776-9df4-c40aaef6eb5a/istio-proxy/0.log"
Sep 14 08:27:36.834789 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:36.834760    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-zdnqg_5f6db21f-387c-48d5-a12f-9f809fbd87d2/istio-proxy/0.log"
Sep 14 08:27:36.839841 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:36.839813    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-zdnqg_5f6db21f-387c-48d5-a12f-9f809fbd87d2/istio-proxy/1.log"
Sep 14 08:27:37.814112 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:37.814078    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5kvk4_ad3788b1-dd2a-4804-bfb8-3ac6b77e03e1/serve-healthcheck-canary/0.log"
Sep 14 08:27:38.329272 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:38.329222    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9xx2k_6c2d43af-8e1f-4a7a-a94d-195c15988364/kube-rbac-proxy/0.log"
Sep 14 08:27:38.355744 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:38.355721    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9xx2k_6c2d43af-8e1f-4a7a-a94d-195c15988364/exporter/0.log"
Sep 14 08:27:38.379248 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:38.379219    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9xx2k_6c2d43af-8e1f-4a7a-a94d-195c15988364/extractor/0.log"
Sep 14 08:27:39.142416 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:39.142391    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-vr64s"
Sep 14 08:27:40.638116 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:40.638080    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-hxw5j_db58eff4-ee8e-497f-a4e7-3e76297e5a7f/manager/0.log"
Sep 14 08:27:40.934863 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:40.934838    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-8mnfx_646ed64e-44c7-4bce-a9fb-0a8ff74e6774/server/0.log"
Sep 14 08:27:41.236865 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:41.236773    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c4c5b687-twcct_1962415f-77ed-420f-afa0-38b3947646ef/manager/0.log"
Sep 14 08:27:41.259085 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:41.259057    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c4c5b687-twcct_1962415f-77ed-420f-afa0-38b3947646ef/copy-manifests/0.log"
Sep 14 08:27:42.040295 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:42.040258    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf_101c9562-855a-4776-9df4-c40aaef6eb5a/istio-proxy/0.log"
Sep 14 08:27:42.487331 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:42.487282    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-zdnqg_5f6db21f-387c-48d5-a12f-9f809fbd87d2/istio-proxy/0.log"
Sep 14 08:27:42.492098 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:42.492074    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-zdnqg_5f6db21f-387c-48d5-a12f-9f809fbd87d2/istio-proxy/1.log"
Sep 14 08:27:42.710797 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:42.710769    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-784955c84b-v8hmn_756e615b-8060-4e08-9d61-81588cb34016/manager/0.log"
Sep 14 08:27:44.132520 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:44.132479    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-wz8sf_101c9562-855a-4776-9df4-c40aaef6eb5a/istio-proxy/0.log"
Sep 14 08:27:44.582565 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:44.582447    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-zdnqg_5f6db21f-387c-48d5-a12f-9f809fbd87d2/istio-proxy/0.log"
Sep 14 08:27:44.587157 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:44.587135    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-zdnqg_5f6db21f-387c-48d5-a12f-9f809fbd87d2/istio-proxy/1.log"
Sep 14 08:27:47.920774 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:47.920745    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-z8xz5_f2032204-d7ba-45f2-81d3-69271762729b/migrator/0.log"
Sep 14 08:27:47.938935 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:47.938906    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-z8xz5_f2032204-d7ba-45f2-81d3-69271762729b/graceful-termination/0.log"
Sep 14 08:27:49.536879 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:49.536848    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rbgj4_214873de-7ce7-4bc5-8817-4e82e603f5d2/kube-multus-additional-cni-plugins/0.log"
Sep 14 08:27:49.555411 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:49.555382    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rbgj4_214873de-7ce7-4bc5-8817-4e82e603f5d2/egress-router-binary-copy/0.log"
Sep 14 08:27:49.579575 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:49.579549    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rbgj4_214873de-7ce7-4bc5-8817-4e82e603f5d2/cni-plugins/0.log"
Sep 14 08:27:49.601645 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:49.601609    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rbgj4_214873de-7ce7-4bc5-8817-4e82e603f5d2/bond-cni-plugin/0.log"
Sep 14 08:27:49.623767 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:49.623741    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rbgj4_214873de-7ce7-4bc5-8817-4e82e603f5d2/routeoverride-cni/0.log"
Sep 14 08:27:49.642819 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:49.642790    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rbgj4_214873de-7ce7-4bc5-8817-4e82e603f5d2/whereabouts-cni-bincopy/0.log"
Sep 14 08:27:49.664973 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:49.664951    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rbgj4_214873de-7ce7-4bc5-8817-4e82e603f5d2/whereabouts-cni/0.log"
Sep 14 08:27:49.869186 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:49.869105    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-tm584_4079005f-d292-4b05-9d42-471d08e7d92a/kube-multus/0.log"
Sep 14 08:27:49.936740 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:49.936706    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4mfj7_fd5d1c55-8829-4084-9b68-36cac615651b/network-metrics-daemon/0.log"
Sep 14 08:27:49.953780 ip-10-0-138-28 kubenswrapper[2562]: I0914 08:27:49.953752    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4mfj7_fd5d1c55-8829-4084-9b68-36cac615651b/kube-rbac-proxy/0.log"